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The  Visual  ’ecnnolody  Research  Simulator  (VTRS)  at  the  Naval  Training 
Equioment  Center  was  used  t.o  study  the  effects  of  six  factors  on  carrier- 
landing  training.  An  in-simulator  transfer  oaradigm  was  chosen  in  which 
students  were  trained  under  various  conditions  and  then  tested  under  a 
standard  condition  that  reoresented  maximum  realism.  The  exoerimental 
design  permitted  a  relatively  larqe  number  of  variables  to  be  studied, 
using  a  relatively  small  number  of  student  subjects.  The  subjects  were 
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pilots  who  had  no  prior  carrier-landing  experience:  16  recent  graduates  of 
Air  Force  T-38  training,  and  16  highly  experienced  Navy  P-3  pilots. 

Display  and  simulator  factors  investigated  were  field  of  view  (160  x  80  vs. 
48  x  36  degrees),  scene  detail  (day,  solid-surface  vs.  night,  point-light) 
and  platform  motion  (six-degrees-of-freedom  vs.  no  motion).  Two  training 
factors  were  included:  descent-rate  cuing  (presence  or  absence  of  an  extra 
element  on  the  Fresnel  lens  display  that  provided  information  on  glideslope 
descent-rate  error),  and  approach  type  (training  on  straight-in  approaches 
vs.  circling  approaches).  Turbulance  was  included  as  a  factor  and  pilot 
type  (Navy  P-3  vs.  Air  Force  T-38)  was  also  included  as  a  factor  to  control 
this  source  of  subject  variability.  After  training  under  a  certain  factor- 
level  combination,  students  were  tested  on  the  day,  wide  field  of  view, 
circling  task  with  motion  and  without  descent-rate  cuing. 

Results  showed  some  temporary  transfer  advantages  for  the  wide  field  of 
view  and  high  scene  detail  conditions.  Training  on  straiqht-in  approaches 
resulted  in  transfer  performance  that  was  better  than  that  produced  by 
training  on  circling  approaches.  There  was  no  motion  or  FLOLS  rate  cuing 
effects  on  the  transfer  task.  Display  and  simulator  transfer  effects  did 
not  differ  between  the  two  pilot  groups  despite  larqe  differences  in  mean 
group  performances. 

As  a  result  of  these  findings,  it  was  suggested  that  a  simulator-to-field 
transfer  study  be  conducted  with  field  of  view,  scene  detail  and  approach 
type  as  factors.  Such  a  study,  using  pilots  from  the  target  population  of 
undergraduate  Naval  aviators,  would  provide  the  necessary  information  to 
make  final  simulator  design  decisions  for  the  carrier-landing  task. 
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SECTION  ! 

INTRODUCTION 

Trie  Visual  technology  Research  Simulator  (VTRS)  at  the  Naval  Training 
Equipment  Center  (NTtC),  Orlando,  Florida,  is  designed  tor  -esearsn  on  fliqnt 
simulator  requirements  for  training  and  skill  maintenance.  The  VTR 5  consists 
of  a  fully  instrumented  Navy  T-2C  jet  trainer  cockpit,  a  ; i x-degree s-of  - 
freedom  synergistic  motion  platform  and  i  de-angl^  visual  system  that  can 
project  computer-generated  (Cl  G }  images  onto  a  spner’cai  screen.  ''he  visual 
system  is  capable  of  displaying  images  viu  target  and  background  projectors 

subtend ina  50  degrees  above  and  30  degrees  below  the  pilot's  eye  level  and  can 

display  160  degrees  of  horizontal  field  (Co  I  Iyer  ana  ‘''hampers,  1973)  . 

The  current  effort  a*  V TRb  involves  research  tu  define  simulator 
requirements  ror  the  carrier  landing  task.  Because  of  trie  large  cost 
implications,  there  is  a  need  to  investigate  a  large  number  of  visual  ana 
other  simulator  features.  A  research  program  was  planned  around  the  holistic 
experimental  philosophy  and  paradigm  proposed  by  Simon  1973;  1977a'  which 
stresses  the  importance  of  studying  as  many  potentially  important  factors  as 
possiole  within  a  single  experiment.  The  researcn  proaram  involves  three 
major  phases.  The  first  phase  consisted  of  performance  experiments  in  whicn 
the  effect  of  various  simulator  components  on  experientea  pilots  in  the 

simulator  was  examined  (westra  et  at.,  1982).  The  second  phase,  descnoea  ’ r 

this  report,  consisted  of  a  quasi-transfer  experiment  in  which  the  simulator 
was  both  the  training  ana  the  criterion  aevice.  Phase  "hree  will  eventually 
include  a  simulator-to-f teld  transfer  experiment  involving  actual  fliont  tests. 

The  information  obtained  from  phase  one  experiments  was  directly  relevant 
to  the  design  of  simulators  for  experienced  pilot  skill  maintenance  ana 
transition  training.  The  inrormation  from  the  phase  two  experiment  reported 
here  is  directly  relevant  to  the  design  of  simulators  for  undergraduate  pilot 
training.  Tne  experiment  involved  pilots  with  no  carrier  landing  experience 
who  were  trained  in  the  simulator  under  various  conditions  representing  levels 
of  several  simulator  equipment  and  training  factors.  The  pilots  were  tnen 
tested  in  the  simulator  in  its  hign  fidelity  conf i gurat ion . 

Three  simulator  equipment  factors  were  studied  in  this  experiment.  A  wide 
field  of  view  was  tested  against  a  narrow  field  of  view  representative  of 
carrier  landing  trainers  in  current  use.  A  daytime  scene  was  compared  to  a 
night  carrier  landing  scene  and  cockpit  motion  was  compared  to  the  absence  of 
motion  cues.  Two  training  factors  were  also  manipulated  during  the  training 
period.  Pi'ots  performed  their  training  trials  with  either  a  conventional 
Fresnel  Lens  Optical  Landing  System  (FLOLS;  or  with  a  Fl.Ol.S  with  added  descent 
rate  information.  Approach  type  was  also  varied  involving  either  a  modified 
straight-in  approach  to  the  carrier  or  a  circling  approach  during  training. 

Two  levels  of  turbulence  were  also  used  durinq  the  train  inn  trials,  with 
pilots  practicing  under  either  calm  or  gusty  wind  conditions. 
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SECTION  II 
EXPERIMENTAL  PLAN 

An  in-simulator  transfer  of  training  paradigm  was  used  to  study  tne 
effects  of  six  factors  plus  a  pilot  type  factor  on  carrier  landing  training. 
Students  were  trained  in  the  simulator  under  various  conditions  representing 
combinations  of  factor  levels,  and  then  tested  under  a  standard  simulator 
configuration  that  represented  maximum  realism.  A  total  of  32  pilots  with  no 
prior  carrier  landing  experience  were  involved  in  the  experiment.  Sixteen 
were  recent  graduates  of  Air  Force  T-38  training  and  16  were  experienced  Navy 
P-3C  pilots. 

FLIGHT  TASK 

The  experimental  task  was  a  simulated  daytime  carrier  approach  and  landing 
on  the  deck  of  the  aircraft  carrier  Forrestal  wi th  a  T-2C  jet.  The  simulated 
carrier  was  moving  at  20  knots  with  a  zero  effective  crosswind  over  the 
ianduig  deck  and  25  knots  relative  wind  down  the  deck.  The  circling  approach 
and  landing  is  depicted  in  Figure  1.  The  task  for  this  experiment  was 
restricted  to  include  only  the  final  turn  of  the  full  circling  maneuver  as 
well  as  the  final  approach  and  landing. 

VISUAL  CUES.  A  successful  carrier  approach  and  landing  involves  the  use  of  a 
family  of  visual  cues  external  to  the  cockpit.  The  principal  cues  come  from  a 
visual  landing  aid  called  the  Fresnel  Lens  Optical  Landing  System  ( F LOLS j  for 
vertical  glideslope  control;  the  carrier  deck  runway,  centerline  and  dropline 
markings  for  lineup  control;  and  the  sky,  horizon,  and  seascape  for  general 
aircraft  attitude  control.  Other  cues  necessary  to  the  operation  of  the 
aircraft,  possibly  including  motion,  are  also  involved  in  carrier  approaches 
and  landings. 

FLOLS.  The  FLOLS  is  the  single  most  important  source  of  information  for  the 
carrier  landing  task.  It  provides  glideslope  displacement  information  to  the 
pilot  during  an  approach.  Physically  located  forward  of  the  LSO  platform  and 
to  the  port  side  of  the  landing  deck  (see  Figure  1),  the  FLOLS  consists  of 
five  Fresnel  lenses  vertically  arranged  between  two  horizontal  light  arrays 
<nown  as  datum  bars.  The  array  of  Fresnel  lenses  provides  an  imaqe  which 
appears  to  the  pilot  as  a  single  sphere  of  light  known  as  the  "meatball." 

This  meatball  is  visible  to  the  pilot  within  a  wedge  of  space  0.75  degree 
aoove  and  below  the  projected  glideslope  of  3.5  degrees  and  plus  or  minus  25 
degrees  horizontally  from  the  center  of  the  wedge  projected  parallel  to  the 
1 anding  deck. 

The  pilot  judges  his  angular  glideslope  deviation  from  the  distance  the 
meatball  appears  to  be  above  or  below  the  datum  bars.  A  meatball  that  appears 
centered  vertically  between  the  datum  bars  indicates  to  the  pilot  that  he  is 
on  the  proper  glidepath.  Figure  2  gives  a  view  of  the  FLOLS  and  its 
projection  aft  of  the  carrier  and  the  perceived  relationship  of  the  meatball 
to  the  datum  bars.  Golovscenko  (1975)  provides  more  detail  on  FLOLS  geometry. 
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MEAT  BALL 

2  MEATBALL  HEIGHTS  ABOVE  CENTER 
-  3/4*  ABOVE  CORRECT  GLIOE  SLOPE 

MEATBALL  CENTERED,  CORRECT  GUDE  SLOPE 


Fiqure  ?_.  Carrier  Approach  Geometry  Depictinq 
FLOLS  Projection  of  Glideslope  Deviation  Information. 
Adapted  from  Golovcsenko  (1976) 
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The  simulated  FLOLS  display  was  a  TV  projection  of  a  C!G  (General 
Electric,  1979)  FLOLS  data  base  defined  by  96  edges.  The  two  datum  oars  that 
normally  consist  of  six  lignts  eacn  were  represented  by  two  solid  bars.  At 
long  range  the  FLOLS  was  magnified  two  times  normal  size,  gradually  shrinking 
to  1.5  times  at  the  ramp.  The  magmf ication  was  required  to  compensate  for 
limited  TV  resolution  so  tnat  the  pi  iots  could  discriminate  meatball  position 
for  glideslope  guidance  at  a  range  similar  to  tnat  in  the  real  world.  This 
FLOLS  magnification  technique  is  used  regularly  on  Navy  carrier  landing 
trainers.  For  example,  the  TA-4J  trainer  (Device  2B3S)  uses  magnifications  as 
large  as  7*. 


FACTORS  AND  LEVElS 

A  large  number  of  factors  potentially  affecting  the  earner  landing  cues 
we-'-e  tentatively  selected  as  candidates  for  study.  These  were  pared  down  by  a 
panel  of  engineers  and  psycho1 ogi sts  into  a  set  of  factors  that  were 
investigated  earlier  in  performance  experiments  (Westra  et  al.,  1982). 

Decisions  regarding  factors  to  be  included  in  this  experiment  were  based 
partly  on  results  from  the  performance  studies.  Other  considerations  were  the 
potential  cost  impact  of  factors  and  the  potential  training  effects  ana 
interactions  involving  factors  falling  in  the  category  of  training  aid  or  type. 

"High"  and  "low"  factor  settings  were  chosen  to  bracket  the  reasonable 
range  of  interest.  For  the  equipment  factors,  the  high  levels  were  generally 
set  at  the  highest  attainable  while  the  lew  levels  were  chosen  to  be  the  most 
degraded  form  of  the  factor  likely  to  be  employed  operationally.  A  summary  of 
the  factors  involved  in  this  experiment  are  given  in  Table  1.  Other  factor 
levels  also  bracket  the  range  of  interest  but  do  not  necessarily  conform  to 
"high"  and  "low"  definitions. 

FIELD  OF  VIEW.  The  high-level  field  of  view  was  a  160-degree  horizontal  by 
80-degree  vertical  display  (Singer-Link,  1977)  which  is  costly  and  is 
repre  .entative  of  that  currently  available  for  carrier  landing  training  only 
on  muiti-task  trainers  such  as  the  2B35  arid  the  F  —14  Wide-Angle  Visual  System 
(WAVS).  The  low-level  field  of  view  was  p'us  or  minus  24  degrees  horizontally 
by  -27  degrees  to  +9  degrees  vertically  which  is  representative  of  the  lower 
cost  Night  Carrier  Landing  Trainers  (NCLTs)  used  for  F-4,  F-14,  A-c,  A-7,  and 
S-3  transition  training. 

SCENE  DETAIL.  The  high  level  of  scene  detail  was  represented  by  a  daytime 
solid-model  CIG  (General  Electric,  1979)  carrier  whose  surfaces  were  defined 
by  985  edges.  The  daytime  scene  included  a  background  with  a  uniform  blue 
below  a  well-defined  horizon.  Brightness  levels  were  approximately  2.80, 

0.50,  and  0.16  fL  for  the  ship,  sea  and  sky,  respectively.  This  level  of 
detail  was  approximately  representative  of  that  available  from  daytime  CIG 
systems  costing  several  millions  of  dollars,  such  as  the  2B35  trainer, 
although  displayed  at  higher  resolution  than  available  in  the  2B35. 
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TABLE  l.  SUMMARY  OF  EX PE "'MENTAL  FACTORS  AND  LEVELS 


LEVEL  SETTINGS 


FACTORS 

"low" 

"h i gn" 

Fieia  of  V i ew 

-2'  degrees  to 

9  degrees  vertical, 
plus  or  minus  ’A 
degrees  nonzont  j ' 

*  -30  degrees  to  co 
decrees  vertical,  plus  or 
minus  °0  degrees  horizonta 

Scene  Detai  1 

Ni qnt :  point-! ight 
sn  i  d 

*  Day:  solid  surface 
snip 

Motion 

F i xed  oase 

♦  Si x-degrees-of  freedom 

Approach  Type 

♦  Circling  aooroacn 

Moo 'tied  straight-i n 
approach 

FlOLS  rate  cuing 

*  Conventional 

FlOLS  display 

FlOLS  o i soiay  wi th 
"command"  rate  cuing 

"Turbulence^- 

Close  to  maximum 
f  lyab I e 

No  turbulence 

Pi  lot  type 

Air  Force  7-38 

Navy  P-3C 

♦Indicates  setting  for  the  transfer  test  configuration. 

^Turbulence  was  set  at  half  the  "low"  level  setting  for  the  transfer  test. 
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The  low  level  of  scene  detail  was  represented  by  an  image  of  a  night 
point-light  CIG  carrier  consisting  of  137  lights.  It  contained  all  deck 
outline,  runway,  centerline  and  drop  lights.  The  background  was  dark  with  no 
visible  horizon.  This  display  is  representative  of  a  night  CIG  system  costing 
less  than  a  million  dollars  and  used  on  several  Navy  NCLTs.  Because  the  scene 
detail  factor  incorporates  elements  of  brightness,  seascape  and  ship  detail, 
it  was  not  fully  crossed  with  field  of  view  and  approach  type  factors, 
although  the  experimental  design  treated  them  as  fully  crossed.  Specifically, 
there  was  no  practical  difference  between  the  wide  and  narrow  field  of  view 
under  the  night  scene  with  straight-in  approaches. 

COCKPIT  MOTION.  A  s i x-degrees-o f-f reedom,  48-inch  synergistic  motion  platform 
(Browder  and  Butrimas,  1981)  was  fully  operational  for  the  high  level  and  was 
stationary  for  the  low  level  of  this  factor.  This  platform  is  similar  to 
those  on  the  Navy's  27  T-2C  Instrument  Trainers  (Device  2F101)  used  in 
Undergr aduate  Pilot  Training  (UPT)  except  that  VTRS  computation  rates  are 
hiqher  for  reduced  cuing  lag.  While  it  is  representative  of  many  older 
platforms  on  existing  trainers,  it  does  not  have  the  low  noise  and  improved 
response  of  new  platforms.  An  attempt  was  made  to  fine-tune  the  operational 
platform  for  optimum  responsi veness  for  the  carrier  landing  task  by  setting 
gains  at  7.5,  2.0,  and  1.2  for  lateral,  vertical,  and  pitch  cuing, 
respecti vely.  Roll,  thrust,  and  yaw  cuing  gains  were  left  at  system  supplied 
1.0  gains. 

APPROACH  TYPE.  Pilots  performed  their  training  trials  with  either  straight-in 
approaches  or  circling  approaches.  For  the  circling  approach  the  aircraft  was 
positioned  abeam  the  ISO  platform  at  5700  feet  from  the  ship  and  at  600  feet 
of  altitude  in  the  approach  conf iquration  (full  flaps,  speed  brake  out,  hook 
and  wheels  down,  and  15  units  angle  of  attack).  Fuel  was  fixed  at  3200  pounds 
to  give  a  gross  aircraft  weight  of  10,000  pounds.  A  trial  consisted  of  the 
final  turn,  approach,  and  landing.  The  straight-in  approach  was  started  with 
the  aircraft  11,990  feet  behind  the  ship  and  4150  feet  to  the  left  of  the 
runway  centerline.  The  initial  altitude  was  400  feet  with  the  aircraft  in  the 
approach  configuration.  Figure  1  depicts  the  two  approach  types.  The 
aircraft  was  trimmed  for  str aight-and-1 evel  flight  in  both  initial  conditions. 

The  straiqht-in  approach  was  defined  specifically  to  provide  an  approach 
with  task  requirements  similar  to  the  circling  approach  but  with  the  ship  in 
view  at  all  times  under  the  narrow  field  of  view  condition.  Thus  the  aircraft 
was  set  to  the  left  of  the  runway  centerline  and  headed  at  18  degrees  to  the 
right  of  the  ship  heading.  Pilots  were  instructed  to  fly  this  modified 
straight-in  approach  straight-and-1 evel  until  approaching  the  runway 
centerline,  then  execute  a  turn  and  roll  out  on  the  runway  centerline. 

If  these  instructions  were  followed,  a  pilot  would  roll  out  for  final 
approach  at  about  3/4-mile  from  the  ship  which  is  similar  to  the  roll  out 
point  from  a  typical  circling  approach.  Further,  if  the  pilot  remained  at 
400  feet  of  altitude  during  the  initial  part  of  the  straight-in  approach,  the 
transition  to  glideslope  descent  would  take  place  just  prior  to  initiating  the 
turn,  necessitating  the  establishment  and  adjustment  of  a  proper  descent  rate 
through  the  latter  part  of  the  turn  as  well  as  final  approach  as  is  the  case 
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with  circling  approaches.  The  initial  distance  from  the  snip  on  the 
straight-in  approach  was  adjusted  sucn  that  flight  times  for  typical  circlinq 
and  straight-in  approaches  were  equal. 

The  approach-type  factor  was  included  in  the  experiment  because  of  the 
evident  possibility  that  the  daytime  mission  can  oe  adequately  taught  with 
(modified)  straight-in  approaches.  This  simplified  procedure  was  supported  by 
results  reported  by  Collyer  et  al.  (1980),  and  by  preliminary  observation.  If 
this  procedure  should  prove  effective,  it  could  have  implications  for  the 
field  of  view  question  since  a  wide  field  of  view  is  not  required  to  keep  the 
ship  in  sight  during  straight-in  approaches. 

FLOlS  RATE  CUING.  The  conventional  version  of  the  FLOLS  display  (see 
Figure  2)  defined  one  level  of  this  factor.  The  other  level  involved  the  use 
of  vertical  pars  displayed  with  the  FLOLS  whicn  presented  glideslope  rate  of 
change  information  to  the  pilots.  This  information  was  presented  to  the 
pilots  in  "command"  fashion,  that  is,  the  bar  height  could  be  interpreted 
directly  in  terms  of  action  required  to  change  to  the  desired  vertical 
velocity.  The  height  of  the  bars  indicated  deviation  from  correct  vertical 
velocity  relative  to  current  glideslope  displacement  and  nulled  bars  indicated 
correct  vertical  velocity  for  that  glideslope  position.  Further  information 
on  the  rate  bars  is  given  in  the  briefing  material  presented  in  Appendix  A. 

The  FLOLS  rate  cuing  factor  was  included  in  the  experiment  on  the  basis  of 
VTRS  work  (Kaul,  Collyer,  and  Lintern,  1980;  Lintern,  Kaul,  and  Sheppard, 

1981)  which  indicated  a  large  improvement  in  glideslope  control  with  rate 
cuing  displays  for  experienced  carrier  landing  pilots.  The  latter  reference 
includes  a  description  of  the  algorithms  used  to  define  the  command  rate  bar 
operation. 

TURBULENCE.  Turbulence  was  included  to  allow  examination  of  simulator  factor 
effects  under  two  difficulty  levels.  The  two  levels,  no  turbulence  and  the 
highest  amount  of  turbulence  under  which  operations  would  continue  at  sea, 
represent  the  range  of  expected  real-world  turbulence.  The  main  effect  of 
turbulence  was  also  of  some  interest  in  this  experiment  as  a  training 
variable.  Turbulence  was  generated  in  the  form  of  pseudo-random  "winds" 
composed  of  various  sinusoidal  frequencies  and  amplitudes  acting  on  the 
longitudinal,  lateral,  and  vertical  aircraft  axis.  The  RMS  values  of  the 
winds  used  were  3.00,  1.25,  and  3.00  ft/sec  for  the  longitudinal,  lateral,  and 
vertical  dimensions,  respectively.  These  values  produced  a  fairly  large 
amount  of  turbulence  judged  by  experienced  carrier  pilots  to  be  near  the 
limits  for  safe  operation.  Frequency  and  amplitude  values  used  are  given  by 
Jewel  et  al .  (1981) . 

PILOT  TYPE.  Pilots  were  selected  from  two  populations  without  carrier  landing 
experience.  One  group  consisted  of  recent  graduates  of  Air  Force  T-38 
traininq,  and  the  other  group  comprised  experienced  Navy  P-3C  pilots.  The 
pilot  group  factor  was  included  explicitly  so  that  the  expected  large  source 
of  subject  variance  resulting  from  differences  between  the  groups  could  be 
directly  estimated  and  removed  from  the  results. 
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FACTORS  HELD  CONS’  ANT.  A  •  v  mer  v‘  f.ict  ’  :i’»~st -.gated  ear  I  ’.er  ( <testra 
et  al. ,  198?'  were  ••ie d  coos: art  '■  this  v aiding  experiment.  Both  CI3  ship 
and  FLOlS  images  were  jieu  i;ir,  ..snout  i;,<:  -.xper  '  nt .  The  "V  line  rate  was 
1025  ana  enqine  computat’cns  we-e  done  *z  30  Hz.  /is  ja 1  lag  was  at  a  system 
best  of  100  msec,  ana  t c-seat  was  oft. 

transfer  conf  igurat 

Af  ter'  comDiCt  -  '  r  i  •r.:ri  ; I  >  ■  *  •:  tr  an  sf  erred  to  an 

in-simulato**  max  :<,\uu  r  •  .  •  sm  ,,i;  ,  -.  •-  •  •.  yt  -,r.e  • rc : i nq  Carr -er  'ancinq 

task.  The  initra1  :•  os  •  t  f  -  t.i.;  -  ir-.  -  c  ibor-'acn  -as  the  s erne  as  that 
used  for  circles:  approach-:-  ■>  tr  a  i  n  :•••  :  .  e  1  o  of  v  :ew  was 

present  along  .*  i  tn  one  •••  -  sce'-e ,  ;  *  ..•= .  on,  -  i-vent  ■  on  a’.  FLOLS  ,  and  an 

intermediate  level  it  t ,.rs i  e-'c  ?.  RM 1  ;es  *or  -re  “woos"  used  during 
training  were  naivec  *  ir  tne  t-ansrer  test.  Tn  is  level  of  turbulence  was 
judged  to  be  somewhere  be twee  *■'  Sin?,  ;  !  \V:  <:  ^  it  . 

PILOTS 

Pilots  were  volunteers  rroo;  r  :g  ,  ■  a"  ions  *•  tn  no  carrier  landing 
experience.  One  group  -f  IF  p ; ?  i s  v-ot:  •:»;  'f  -ecant  a-iduates  of  Air 
Force  T  —38  training,  a;  1  toe  pilots  in  tiis  -  >.:.  n  approximately  200  hours 
of  total  flight  time  with  ar>out  100  nour<  v;  *  1 1 oh',  -.ine  in  the  "-23  in  the 
six  months  prior  to  t.->e  experiment.  To >■  r -u-r  :r ■; ;. comprised  16  experienced 
Navy  °-3C  pilots.  This  roup  aver  ace  j  i^u'-s  r  teta’  riiqht  time  Put 

varied  consider  ,ibi  y  i  n  ivur  .  •  -xoer  icoou  •<  t  ■  .  :  tar  care  deviation  of  529  and 
a  range  of  600  to  I'-r-'.  f i .wc  ho  .r  s.  !•  /.••••••'ence  in  trie  s' x  mentns  preceding 

the  experiment  av-.r  idee  hom-s,  a . t  re  -C,  with  a  standard  deviation 
of  105  and  a  range  pf  50  to  3?C-  f  1 :  or.t  hc^r  ; . 

PERFORMANCE  MEASURED 

All  of  the  summary  "{.-a  seres  col  '-ecte  i  a  no  des<.  ribed  by  «estra  et  al . , 
(1982)  were  also  co •  lec for  the  present  ixper^nrent  However,  only  several 
basic  measures  that  summ.vize  final  apero-ion  and  1 anoing  outcome  were  used  in 
analysis  presented  he-e.  Percent  time  or  trrnet  summary  measures  for  the 
final  approach  were  u  ,eo  r.jther  than  tne  v.-re  *- 1  ltr.n  ally  lesirable  RMS 
error  scores  to  deal  with  the  problem  of  rijn-erous  "r,-.  t  i  ier"  trials  by  the 
novice  pilots.  The  time  an  target  neasur-  effectively  limits  a  score  and  the 
problem  of  extensive  data  editing  for  oarers  i  s  reduced.  An  accuracy  ’core 
for  touchdown  pert  ormatii.e  n  the  long  1  :i men  si  on  wa,;-  used  to  assess 

landing  performance.  This  score  was  esseritnily  the  inverse  of  absolute  error 
from  the  desired  touchdown  point  w’th  i  in-  it  for  ceaimq  with  missed  i  Hidings. 

APPROACH  SCORES.  Percent,  time  on  target  ( TOT)  summary  scores  were  calculated 
for  the  segment  3000  ft.  to  ''imp.  [n  the  ''ase  of  circling  approaches, 
measurement  was  net  started  until  -«.i  i  out  n.i,:  tecurre  j  so  that  the 
measurement  segment  was  less  than  TOGO  ft.  for  some  circling  approaches  with 
tight  turns.  The  roll  out  point:  was  defined  -,s  the  point  at  which  the 
aircraft  heading  hao  crossed  Mie  sh.  l  p '  s  hea.l’n  j  and  aircraft  bank  was  within 
four  degrees  of  level.  Measures  or  * .nree  variables  describing  aircraft 
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position  relative  to  the  defined  optimum  flight  path  were  used  in  computing 
the  TOT  scores: 

•  Vertical  deviations  in  degrees  from  tne  specified  glideslope; 

•  Horizontal  deviations  in  degrees  from  the  center  of  the  lanoing  dec* 
(lineup);  and 

•  Angular  deviations,  in  units  from  the  optimum  angle  of  attack. 

Based  on  recommendations  by  Navy  Landing  Signal  Officers  (LSOs),  tolerance 
Danas  describing  operationally  desired  performance  were  set  at  plus  or  minus 
0.3  degree,  plus  or  minus  1.0  degree,  and  plus  or  minus  1.0  unit  for 
glideslope,  lineup,  and  angle  of  attack,  respectively.  The  tolerance  for 
glideslope  represents  approximately  plus  or  minus  one  "meatball1'  of  deviation 
of  the  FLOLS  display.  The  percent  TOT  scores  are  referred  to  as  glideslope, 
lineup,  and  angle  of  attack  tracking  scores  in  the  discussions  tnat  follow. 

T0UCH00WN  WIRE  ACCURACY  SCORE.  A  score  reflecting  touchdown  wire 
(longitudinal)  accuracy  was  created  for  assessing  touchdown  quality.  This 
score  is  similar  to  Brictson's  (1973)  Landing  Performance  Score  (LPS),  out  it 
is  based  on  absolute  deviation  from  the  desired  touchaown  point  rather  than 
wire  catch  per  se.  This  gives  the  score  better  statistical  properties  and 
more  appropriately  assesses  quality  for  number  one  wire  traps  and  oolters. 

With  the  LPS,  all  one-wire  catches  get  the  same  score  while  a  considerably 
different  wire  accuracy  score  can  be  achieved  depending  on  whether  the 
touchdown  is  well  short  of  the  one-wire  or  very  near  the  one-wire. 

The  algorithm  used  to  create  the  wire  accuracy  score  is  depicted  in 
Figure  3.  The  score  is  essentially  based  on  the  wires  with  the  distance 


Touchdown 
W  i  re 

Accuracy 

Score 


Ramp  33  132  165  198  231  330 

Hook  Touchdown  Distance  from  Ramp 
on  Landing  Deck  (feet) 


Figure  3.  Scoring  Algorithm  for  Touchdown  Longitudinal  Accuracy 
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between  the  w’res  used  as  a  "unit  of  measure."  A  tnree  wire  catch  results  in 
a  "perfect"  score  of  100  while  a  touchdown  tnree  wire-distance-units  or  more 
past  the  four  wire  or  three  wire  distance  units  or  more  behind  the  one  wire 
results  in  a  score  of  zero.  Beyond  these  limits  a  landing  is  in  unsafe, 
near-crash  conditions,  and  would  almost  certainly  have  been  waved  off  had  an 
ISO  been  in  control  of  the  flight.  Touchdowns  more  than  one  wire-distance- 
unit  behind  the  one  wire  or  more  than  one  wire-distance-unit  beyond  the  four 
wire  would  also  often  have  been  waved  off  at  sea.  However,  differential 
scoring  in  the  "marginal"  landing  areas  was  helpful  for  assessment  purposes  in 
this  experiment  in  which  no  waveoffs  were  given,  and  pilots  were  instructed  to 
attempt  a  landing  unless  a  crash  were  imminent. 

COVARIATE  TASK 

Oata  were  collected  from  all  subjects  on  an  ATARI  video  game  sailed  Air 
Combat  Maneuvering  (ACM)  which  is  commercially  available  (Combat  Tape  CX2601, 
Game  No.  24,  difficulty  "b",  right  controller).  ACM  is  a  game  involving  two 
moving  aircraft  displayed  on  a  television  screen.  One  of  the  aircraft  is 
controlled  by  the  subject  with  a  control  stick  to  turn  right  or  left  and  to 
speed  up  or  slow  down.  The  other  aircraft  is  the  target  which  tracks  on  a 
constant  heading  after  the  initial  neading  is  randomly  selected  following  each 
"hit."  The  subject  also  controls  a  trigger  for  a  missile  on  his  aircraft 
which  is  aligned  with  the  aircraft's  longitudinal  axis  and  fired  at  the  target 
aircraft.  The  missile  heading  can  also  be  controlled  by  the  subject  to  a 
limited  degree  after  it  leaves  the  aircraft.  A  hit  is  scored  when  the  target 
aircraft  is  struck  by  a  missile  from  the  subject's  aircraft.  A  subject's 
score  for  one  game  is  the  total  number  of  hits  during  a  2.25-minute  trial 
period. 

It  was  hypothesized  that  scores  on  ACM  would  predict  carrier  approach 
glideslope  tracking  ability  since  the  game  involves  reaction  time,  aiming,  and 
tracking  skills  that  are  basic  to  carrier  landing  success.  Kennedy,  Bittner 
and  Jones  (1981)  reported  a  strong  relationship  between  terminal  performance 
on  ACM  and  terminal  performance  on  a  compensatory  tracking  task  (r=.78). 
Further,  other  research  with  this  game  suggested  that  it  had  a  number  of 
characteristics  desirable  in  a  covariate  (Jones,  Kennedy  and  Bittner,  1981). 

The  game  is  easy  to  administer,  low  in  cost,  easily  transportable,  and  is  high 
in  intrinsic  reward  value.  But  more  importantly,  performance  stabilizes  early 
(correlations  over  time  are  constant  after  a  short  acquisition  period)  with 
high  "task  definition"  (correlations  among  stabilized  trials  are  nigh,  r  =  .9). 

PROCEDURES 

Each  pilot's  experiment  sequence  consisted  of  40  training  trials  and  16 
transfer  trials.  Before  flying  any  trials,  pilots  were  given  approximately 
1.5  hours  of  instruction  in  carrier  landing  procedures  in  the  form  of  a 
briefing  and  an  instructional  videotape.  They  then  flew  two  three-minute 
familiarization  flights  in  the  simulator  before  commencing  with  the 
experimental  training  trials.  Instructional  feedback  was  given  after  each 
training  trial  by  a  VTRS  staff  member  and  an  automatic  ISO  (to  be  described) 
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was  used  throughout  the  experiment  to  give  calls  during  the  flights. 
Instructional  feedback  was  not  given  during  the  transfer  test  trials. 

BRIEFING.  Pilots  were  given  a  pamphlet  containing  basic  information  on 
carrier  landing  procedures  as  well  as  any  additional  information  necessary  for 
a  pilot's  particular  training  condition.  For  example,  pilots  who  were 
assigned  to  a  training  condition  with  rate  bars  on  the  FiOlS  received 
additional  information  regarding  the  function  and  use  of  the  rate  bars.  The 
complete  set  of  briefing  material  is  given  in  Appendix  A.  Pilots  retained 
their  briefing  material  throughout  the  experiment  and  were  encouraged  to  refer 
to  the  material  at  any  time  while  not  engaged  in  flying. 

The  preliminary  briefing  itself  consisted  of  a  verbal  "walk  through"  of 
the  pamphlet  with  emphasis  on  the  key  procedures.  This  briefing  was  conducted 
by  a  member  of  the  VTRS  research  staff.  Members  of  the  VTRS  research  staff 
authored  the  briefing  pamphlet  with  assistance  from  several  naval  aviators  ana 
LSOs.  Material  for  the  pamphlet  was  drawn  from  Navy  carrier  landing 
instruction  manuals  and  documents  including  Landing  Signal  Officer  (1975), 

FCLP  Pattern  (1977),  Carrier  Qualification  Procedures  (1977),  and  Flight 
Training  Instruction  (1979). 

INSTRUCTIONAL  VIDEOTAPE.  Following  the  verbal  briefing,  pilots  were  shown  a 
portion  of  a  Navy  film  describing  key  carrier  landing  procedures  which  is  also 
shown  to  undergr aduate  naval  aviators  during  their  training.  This  film 
entitled  "Carrier  Landing  Mishaps"  is  available  through  the  Navy  film  library. 

FAMILIARIZATION  FLIGHTS.  A  briefing  on  the  T-2C  cockpit  and  controls  was 
given  prior  to  the  first  flight.  Pilots  were  then  given  two  three-minute 
familiarization  flights  prior  to  their  experimental  training  trials.  Initial 
aircraft  positions  were  set  in  accordance  with  the  training  condition  to  which 
the  individual  pilots  had  been  assigned.  However,  all  familiarization  flights 
were  flown  with  no  turbulence,  no  motion,  and  a  wide  field  of  view. 

Pilots  were  instructed  to  fly  to  the  carrier  from  their  initial  positions, 
attempt  to  locate  the  FLOLS,  start  a  descent,  and  then  fly  over  the  carrier 
without  attempting  to  land.  They  were  then  instructed  to  make  a  left  turn  and 
circle  for  another  approach  after  flying  over  the  carrier  and  to  continue  for 
three  minutes.  Pilots  were  tola  to  "exercise"  the  controls  and  to  try  to 
familiarize  themselves  with  the  cockpit  and  controls  as  much  as  possible 
during  the  preliminary  flights.  If  a  crash  occurred,  the  simulator  was  reset 
to  the  appropriate  initial  position,  and  elapsed  time  was  subtracted  from 
remaining  time  so  that  all  pilots  had  exactly  six  minutes  of  familiarization 
f 1 i ght . 

AUTOMATIC  LSO.  An  automatic  LSO,  giving  calls  via  a  VOTRAX  '/oice  generation 
system,  was  used  for  all  trials  in  the  experiment.  The  automatic  LSO  software 
was  developed  from  a  modified  version  of  the  model  described  by  Borden  and 
McCauley  (1978).  A  restricted  set  of  calls  from  the  complete  software  proqram 
was  used  to  avoid  overloading  the  trainees  with  information.  All  calls 
pertaining  to  lineup  were  eliminated  from  the  LSO  repertoire.  Only  a  set  of 
basic  glideslope  and  angle  of  attack  calls  was  retained,  and  although  angle 
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of  attack  calls  were  available,  they  were  rarely  given  because  of  low  priority 
in  the  program  logic.  Implicit  in  this  use  of  the  automatic  LSO  was  an 
emphasis  on  glideslope  control  during  the  experiment. 

Additionally,  no  waveoffs  were  given,  and  pilots  were  instructed  to  try  to 
land  on  every  approach,  aborting  only  if  a  crash  were  imminent.  If  a  pilot 
exceeded  the  automatic  LSO's  waveoft  criterion  limits  during  an  approach,  the 
ISO  simply  shut  off  until  such  time  as  the  pilot  got  back  within  waveoff 
limits,  unless  the  approach  were  within  2000  feet  of  the  ramp,  in  which  case 
the  LSO  stayed  off. 

INSTRUCTIONAL  FEEDBACK.  Instructional  feedback  was  biven  after  every  training 
trial.  This  feedback  was  given  by  two  members  of  the  VTRS  staff  familiar  with 
the  carrier  landing  task.  The  "instructors"  attempted  to  give  feedback  that 
was  as  uniform  as  possible,  and  instructors  rotated  every  10  or  20  trials  such 
that  any  instructor  effect  was  balanced  across  pilots.  The  feedback 
instruction  philosophy  was  to  identify  the  major  problem  or  error  that 
occurred  during  a  trial  and  then  advise  as  to  the  proper  error  correction 
procedure. 

The  instructors  received  preliminary  training  from  a  Navy  LSO  and  drew 
heavily  from  the  Navy  publications  referenced  earlier  for  descriptions  of 
error  correction  procedures.  Emphasis  was  placed  on  glideslope  correction 
procedures  more  or  less  "by  the  book"  except  that  use  of  the  throttle  for 
glideslope  tracking  was  heavily  emphasized  to  overcome  pilot  tendencies  to 
track  the  glideslope  with  elevator  (nose)  control.  References  to  lineup  and 
angle  of  attack  problems  were  rarely  made  except  in  a  few  cases  in  which 
severe  problems  were  noted. 

Pilots  flying  circling  approaches  during  training  were  also  given  feedback 
on  turn  parameters  including  bank  angle  control,  angle  of  attack  control, 
overshoot,  undershoot,  long  in  the  turn,  short  in  the  turn,  and  altitude 
control.  Pilots  flying  modified  straight-in  training  approaches  were  given 
feedback  regarding  their  control  during  the  initial  straight-and-1 evel  portion 
and  their  roll  out  for  lineup.  Instructors  were  aided  considerably  in  their 
ability  to  give  feedback  by  a  CRT  graphical  display  that  provided  plots  of 
glideslope  deviation,  lineup  deviation,  angle  of  attack  deviation,  vertical 
velocity,  throttle  position  and  aircraft  pitch  for  the  final  approach. 

SCHEDULING.  Pilots  performed  the  experiment  in  pairs,  starting  on  either  a 
Monday  or  a  Wednesday.  Pairs  starting  on  Monday  flew  20  training  trials  each 
Monday,  20  on  Tuesday,  and  then  16  transfer  trials  on  Wednesday.  Pairs 
starting  on  Wednesday  flew  10  training  trials  on  Wednesday,  20  training  trials 
on  Thursday,  and  10  training  trials  and  16  transfer  trials  on  Friday.  A  pilot 
performed  ten  consecutive  training  trials  in  a  single  session  and  then 
alternated  with  the  other  pilot  in  his  pair  until  the  day's  schedule  was 
complete.  A  single  training  session  took  about  45  minutes  to  complete. 
Transfer  trials  were  flown  in  similar  alternating  sessions  except  that  there 
were  eight  trials  per  session  rather  than  ten. 
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Data  on  the  ATARI  Air  Combat  Maneuvering  (ACM)  video  game  were  collected 
in  sessions  of  ten  trials  each  from  a  pilot  when  the  other  pilot  in  his  pair 
was  flying  the  simulator.  The  first  16  pilots  in  the  experiment  performed  50 
2.25-minute  ACM  trials  while  the  second  16  performed  30  trials  after  evidence 
from  the  first  16  pilots  indicated  30  trials  were  sufficient  for  covariate  use 
of  the  data. 

EXPERIMENTAL  DESIGN 

A  transfer  of  training  paradigm  was  superimposed  on  the  basic  experimental 
design  which  was  an  adaptation  of  National  Bureau  of  Standards  (1957)  Plan 
4.7.16.  Each  pilot  performed  40  training  trials  under  one  of  the  conditions 
of  the  basic  design  followed  by  16  transfer  trials  in  the  simulator  on  the 
high  fidelity  transfer  configuration.  The  basic  design  was  1/4  of  a  fully 
crossed  2?  design  resulting  in  32  experimental  conditions.  The  defining 
generators  and  generalized  contrast  that  define  the  experimental  conditions 
and  alias  structure  (see  Box  and  Hunter,  1961;  Davies,  1967;  Simon,  1973; 
1977b)  are:  I  =  A8CEG  =  A8DF  =  CDEFG,  where  the  factors  field  of  view, 
motion,  pilot  type,  scene  detail,  turbulence,  FLOLS  rate  cuing,  and  approach 
type  are  associated  with  labels  A,  B,  C,  0,  E,  F,  and  G,  respectively. 

All  main  effects  are  confounded  only  with  three-way  or  higher  interactions 
as  are  15  of  the  21  two-way  interactions.  The  other  six  two-way  interactions 
are  confounded  in  strings  of  two  each  and  the  remaining  six  estimable  terms  in 
the  basic  design  represent  strings  of  three-way  interactions.  As  the  basic 
design  was  repeated  across  trials,  trial  effects  are  also  fully  estimable. 

The  two-way  interactions  judged  least  likely  to  be  important  a  priori  were 
assigned  to  the  strings  of  confounded  two-factor  interactions.  Implicit  in 
the  use  of  this  fractional  factorial  design  is  the  assumption  that  three-way 
and  higher-order  interactions  will  generally  be  negligible.  Each  of  the 
estimable  effects  is  also  confounded  with  the  subject  effects  defined  by  the 
groupings  involved  in  a  contrast,  and  for  this  reason  a  covariate  task  was 
included  in  the  experiment. 
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SECTION  II  I 
RESULTS 

Figures  4  through  7  graphically  describe  the  experimental  results  for  main 
effects.  The  numbers  in  parentheses  above  a  block,  of  trials  for  the  effects 
represent  the  results  of  an  analysis  of  variance  for  that  block  of  trials. 

The  numbers  are  the  ranks  of  the  sizes  of  the  effects  (within  the  31  estimable 
effects)  of  the  basic  design.  Examination  of  Figure  4,  for  example,  indicates 
that  in  the  first  block  of  training  trials  turbulence  had  the  largest  effect, 
approach  type  ranked  second,  motion  ranked  tenth,  etc.  An  effect  can  rank 
high  by  chance  alone,  of  course,  and  such  things  as  stability  over  time  and 
effect  size  must  also  be  taken  into  account  when  judging  the  meam ngf u 1  ness 
and  reliability  of  an  effect.  On  the  other  hand,  if  an  effect  size  does  not 
rank  very  high,  it  probably  is  neither  meaningful  nor  real  since  it  cannot  be 
differentiated  from  noise. 

Tables  2  through  5  present  analysis  of  variance  summaries  of  transfer  test 
trials  for  performance  scores  shown  in  Figure  4  through  7.  These  analyses 
estimate  effects  averaged  across  all  16  transfer  trials,  represented  by  two 
blocks  of  eight  trial  means.  Main  effects  were  tested  separately  as  were  the 
field  of  view  by  approach  type  and  scene  detail  by  approach  type  interactions 
which  were  considered  important  on  a  priori  grounds.  Graphical  results  for 
these  two  interactions  are  given  in  Figures  8  and  9.  Analysis  of  variance 
summaries  for  training  trials  are  presented  in  Appendix  B  and  are  generally 
not  discussed.  Results  for  the  covariate  task  and  the  implications  of  these 
results  are  presented  and  discussed  separately. 

The  basic  "residual"  term  for  an  analysis  of  variance  was  created  from  the 
sum  of  the  two-  and  three-way  string  terms.  The  sum  of  two-way  interactions 
not  involving  pilot  type  and  the  sum  of  the  two-way  interactions  involving 
pilot  type  were  tested  against  the  basic  residual  term.  If  these  effects  were 
not  significant,  they  were  combined  with  the  basic  residual  term  to  form  a 
residual  estimate  against  which  all  other  effects  were  tested.  Thus,  for 
example,  the  F-ratios  shown  for  the  single-degree-of-freedom  effects  in 
Table  2  have  a  22-degree-of-freedom  denominator  whose  sums  of  squares  is  from 
all  the  indicated  sources.  There  is  an  exception  to  this  in  Table  5  for  angle 
of  attack  tracking  scores  in  which  the  combined  two-factor  interaction  term 
not  involving  pilots  was  significant.  In  this  case  there  were  two  fairly 
large  interactions  involving  turbulence,  but  because  of  problems  in  the 
interpretation  of  turbulence  effects,  the  whole  term  was  added  to  the  residual. 

The  entire  transfer-of-traini ng  design  may  be  thouqht  of  as  a  special  case 
of  a  repeated  measures  design  (Winer,  1971,  ch.  7)  with  observations  repeated 
on  a  trials  factor.  In  this  sense,  the  residual  estimate  represents  an 
estimate  of  subjects  within  groups  (between  subjects)  error.  Trial  blocks 
were  tested  against  a  term  comprised  of  all  estimable  three-way  and  hiqher 
interactions  involving  blocks  which  is  then  an  estimate  of  error  within 
subjects.  Two-factor  interactions  involving  blocks  were  tested  omnibus 
fashion  against  this  same  term.  Estimable  two-factor  interactions  involving 
trials  were  also  examined  individually,  but  in  the  absence  of  strong  evidence 
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Figure  4.  Main  Effects  for  Touchdown  Wire  Accuracy  Score. 
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Figure  5. 
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Main  Effects  for  Glideslope  Tracking  Score: 

Percent  Time  +  0.3°  of  Desired  for  3000  Feet  to  Ramp. 
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Figure  6.  Main  Effects  for  Lineup  Tracking  Score: 

Percent  Time  +  1.0°  of  Desired  for  3000  Feet  to  Ramp. 
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Figure  7.  Main  Effects  for  Angle  of  Attack  Tracking  Score: 

Percent  Time  +  1.0  un.  of  Desired  for  3000  Feet  to  Ramp. 
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TABLE  2.  ANALYSIS  OF  VARIANCE  FOR  TOUCHDOWN 
WIRE  ACCURACY  TRANSFER  SCORES 


i 


i 


LEVELS  Mean 


Source  of  Variance 

High 

Low 

df 

Difference^ 

F 

Field  of  View 

Wide 

Nar 

1 

0.4 

(-)2 

0.01 

Scene  Detai 1 

Day 

Ni  ght 

1 

1.6 

(-) 

0.13 

Motion 

On 

Off 

1 

4.8 

(2.5) 

1.60 

Approach  Type 

St.  In 

Circ 

1 

0.0 

(-) 

0.00 

FLOLS  Rate  Cue 

Cui  ng 

No  Cue 

1 

2.3 

(-) 

0.36 

Turbulence 

Calm 

Winds 

1 

5.0 

(2.7) 

1.75 

Pilot  Type 

Nav  P-3C 

AF  T38 

1 

-13.3 

(19.1) 

12.13** 

FOV  x  App.  Type 

1 

(1-0) 

0.61 

S.OTL  x  App.  Type 

1 

(-) 

0.43 

2-Factor  In t  (No  Pi  1 ) 3 

7 )  Re— 

(7.0) 

0.44 

2-Factor  Int  (Pi  1 )4 

6 ) s i d- 

(7.4) 

0.55 

2*3  Way  Strings 

9)ual 

(20.2) 

31ocks  (3  Trials) 

1 

4.9 

(2.6) 

1.84 

2-Factor  Int  (Blocks) 

7 

(2.4) 

0.24 

3-Factor  Int  (Blocks) 

24 

(33.8) 

(within  subject  residual) 

Grand  Mean 

39.5 

Std.  Err.  Difference 

3.8 

Std.  Deviation 

10.8 

1  iMean  of  observations  taken  under  high  level  minus  mean  of  observations  taken 

under  low  level  of  factor. 

j 

|  2ya  1  ues  in  parentheses  are  percent  variance  accounted  for  in  the  experiment. 

Percents  less  than  1.0  are  shown  by  a  dash  (-). 

| 

;  3jwo_factor  interactions  not  involving  pilot  type. 

|  4fwo-f actor  interactions  involving  pilot  type. 

•  *p  <_  .05 

!  **p  <  .01 

1 

J 

i 


1 


24 
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.  TABLE  3.  ANALYSIS  OF  VARIANCE  FOR  GLIDESLOPE 
TRACKING  TRANSFER  SCORES 


LEVELS 

Mean 

Source  of  Variance 

High 

Low 

df 

Difference! 

F 

Field  of  View 

Wide 

Nar 

1 

6.3 

(8.4)2 

6.42* 

Scene  Oetai 1 

Day 

Ni  ght 

1 

4.6 

(4.6) 

3.51 

Motion 

On 

Off 

1 

3.2 

(2.2) 

1.68 

Approach  Type 

St.  In 

Circ 

1 

2.5 

(1.4) 

1.07 

FLOLS  Rate  Cue 

Cuing 

No  Cue 

1 

-3.9 

(3.3) 

2.52 

Turbulence 

Ca  lm 

Winds 

1 

2.8 

(1.6) 

1.22 

Pilot  Type 

Nav  P-3C 

AF  T38 

1 

-7.6 

(12.3) 

9.40** 

FOV  x  App.  Type 

1 

(-) 

0.46 

S.DTL  x  App.  Type 

1 

(-) 

0.46 

2-Factor  Int  (No  Pi  1)3 

7)Re- 

(9.7) 

1.13 

2-Factor  Int  (Pi  l)4 

6)sid- 

(8.1) 

1.10 

2*3  Way  Strings 

9)ual 

(11.0) 

Blocks  (8  Trials) 

1 

5.9 

(7.5) 

8.42** 

2-Factor  Int  (Blocks) 

7 

(7.4) 

1.18 

3-Factor  Int  (Blocks) 

24 

(21.4) 

(within  subject  residual 

) 

Grand  Mean 

34.8 

Std.  Err.  Difference 

2.3 

Std.  Deviation 

6.5 

iMean  of  observations  taken  under 

high  level 

minus 

mean  of 

observations 

taken 

under  low  level  of  factor. 


^Values  in  parentneses  are  percent  variance  accounted  for  in  the  experiment. 
Percents  less  than  1.0  are  shown  by  a  dash  (-). 

^Two-f actor  interactions  not  involving  pilot  type. 

4Two-f actor  interactions  involving  pilot  type. 

*p  <  .05 

**p  <  .01 


i 


L 
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TABLE  4.  ANALYSIS  OF  VARIANCE  FOR  LINEUP 
TRACKING  TRANSFER  SCORES 


LEVELS  Mean 


Source  of  Variance 

Hi  gh 

Low 

df 

Difference^ 

F 

Field  of  View 

Wide 

Nar 

1 

9.9 

(5.0)2 

3.03 

Scene  Oetai 1 

Oay 

Ni  ght 

1 

5.4 

(1.4) 

0.88 

Motion 

On 

Off 

1 

4.8 

(1.2) 

0.71 

Approach  Type 

St.  In 

Circ 

1 

15.3 

(11.7) 

7.14* 

FLOLS  Rate  Cue 

Cuing 

No  Cue 

1 

-7.5 

(2.8) 

1.70 

Turbulence 

Ca  1m 

Winds 

1 

-1.4 

(-) 

0.05 

Pilot  Type 

Nav  P-3C 

AF  T38 

1 

21.5 

(23.1) 

14.08** 

FOV  x  App.  Type 

1 

(-) 

0.14 

S.DTL  x  App.  Type 

1 

(1.1) 

0.63 

2-Factor  Int  (No  Pi  1)3 

7)Re- 

(9.8) 

1.03 

2-Factor  Int  (Pi  1)4 

6 )  s i d— 

(14.1) 

1.72 

2*3  way  Strings 

9)ua1 

(12-2) 

Blocks  (8  Trials) 

1 

(1.7) 

2.96 

2-Factor  Int  (Blocks) 

7 

(1.7) 

0.42 

3-Factor  Int  (Blocks) 

24 

(14.0) 

(within  subject  residual 

) 

Grand  Mean 

64.7 

Std.  Err.  Difference 

5.6 

Std.  Deviation 

15.9 

iMean  of  observations  taken  under  high  level  minus  mean  of  observations  taken 
under  low  level  of  factor. 

^Values  in  parentheses  are  percent  variance  accounted  for  in  the  experiment. 
Percents  less  than  1.0  are  shown  by  a  dash  (-). 

^Two-factor  interactions  not  involving  pilot  type. 

^Two-f actor  interactions  involving  pilot  type. 

*p  <  .05 
**p  <  .01 
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TABLE  5.  ANALYSIS  OF  VARIANCE  FOR  ANGLE  OF  ATTACK 
TRACKING  TRANSFER  SCORES 


LEVELS  Mean 


Source  of  Variance 

Hi  qh 

Low 

df 

Difference* 

F 

Field  of  View 

Wide 

Nar 

1 

10.8 

(13. 0)2 

8.09** 

Scene  Detail 

Day 

Ni  ght 

1 

5.4 

(3.3) 

2.04 

Motion 

On 

Off 

1 

2.0 

(-) 

0.28 

Approach  Type 

St.  In 

Circ 

1 

-0.1 

(-) 

0.00 

FLOLS  Rate  Cue 

Cuing 

No  Cue 

1 

-4.4 

(2.2) 

1.36 

Turbulence 

Calm 

Winds 

1 

-3.3 

(1.2) 

0.76 

Pi  lot  Type 

Nav  P-3C 

AF  T38 

1 

13.0 

(18.8) 

11.75** 

FOV  x  App.  Type 

1 

(2.5) 

1.57 

S.DTL  x  App.  Type 

1 

(-) 

0.15 

2-Factor  Int  (No  Pi  1)3 

7)Re- 

(20.2) 

3.08 

2-Factor  Int  (Pi  l)4 

6 )  s  i  d- 

(6.6) 

1.18 

2*3  Way  Strings 

9)ual 

(8-4) 

Blocks  (8  Trials) 

1 

2.9 

(-) 

1.39 

2-Factor  Int  (Blocks) 

7 

(6.9) 

1.53 

3-Factor  Int  (81ocks) 

24 

(15.4) 

Grand  Mean 

45.1 

Std.  Err.  Difference 

3.8 

Std.  Deviation 

10.8 

*Mean  of  observations  taken  under 

high  level 

minus 

mean  of 

observations 

taken 

under  low  level  of  factor. 


2va  lues  in  parentheses  are  percent  variance  accounted  for  in  the  experiment. 
Percents  less  than  1.0  are  shown  by  a  dash  (-). 

^Two-factor  interactions  not  involving  pilot  type. 

4Two-f actor  interactions  involving  pilot  type. 

*p  <  .05 

**p  <  .01 
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Fiqure  8.  Results  Tor  Field  of  View  by  Approach  Type. 
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Figure  9.  Results  for  Scene  Detail  by  Approach  Type 
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tor  an  effect,  they  were  not  partial  led  out  of  the  combined  te^ms  and  are  not 
discussed . 

OTHER  INFORMATION.  There  are  certain  computations  that  can  be  performed  to 
obtain  supplementary  information  with  which  to  interpret  the  data.  The 
numbers  required  for  these  analyses  are  available  in  the  tables. 

The  mean  performance  for  high  and  low  levels  of  any  factor  can  be  obtained 
by  taking  the  grand  mean  shown  at  the  bottom  of  each  column  and  to  it  add 
(high  level)  or  subtract  (low  level)  half  of  the  mean  difference  for  that 
factor.  The  siqn  of  the  mean  difference  must  be  taken  into  consideration  in 
this  calculation  since  the  mean  of  the  low  condition  was  always  subtracted 
from  the  mean  of  the  high  condition  to  obtain  the  mean  difference.  A  positive 
percent  t ime-on-target  mean  difference  indicates  better  performance  with  the 
factor's  higher  level. 

Confidence  limits  for  the  mean  differences  can  be  roughly  obtained  by 
multiplying  the  standard  error  of  the  mean  difference — "STD  ERR  DIFF"  at  the 
Dottom  of  the  table--by  plus  or  minus  two  for  the  95  percent  level  and  plus  or 
minus  2.6  for  the  99  percent  level.  These  values  can  be  added  to  the  mean 
differences  of  each  factor  to  obtain  the  low  and  high  limits  within  which  the 
true  mean  difference  is  expected  to  lie.  Standard  deviations  are  based  on 
subject  within  group  error  estimates  and  standard  errors  of  differences  are 
based  on  16  subjects  per  group. 

F-r at i os  for  a  particular  effect  can  be  calculated  using  the  percentages 
in  the  table.  The  numerator  of  the  ratio  is  the  percent  variance  accounted 
for  by  an  effect  divided  by  its  degrees-of-freedom,  and  the  demoninator  is  the 
residual  percent  variance  accounted  for  divided  by  its  degrees-of-freedom. 
Significance  is  indicated  at  .05  and  .01  levels.  For  these  results  it  is 
suggested  that  .01  is  an  appropriate  level  of  significance  providing  some 
compensation  for  the  multiple  tests  and  measures  involved.  On  the  other  hand, 
because  of  the  small  number  of  degrees-of-freedom  available  for  "error" 
esimation,  only  large  effects  are  likely  to  show  up  as  significant  at  the  .01 
leve  1 . 

INTERPRETATION  STANDARDS.  That  an  observed  difference  between  two  conditions 
is  or  is  not  statistical ly  significant  provides  little  information  regarding 
the  practical  significance  of  the  difference.  Some  outside  "real-world" 
standards  are  needed  to  evaluate  the  data.  The  TOT  scores  are  of  some  value 
themselves  in  this  regard  since  they  can  be  interpreted  directly  in  terms  of 
percent  time  within  "real-world"  defined  levels  of  good  performance.  Scores 
of  100  for  an  approach  represent  "perfect"  performance  from  an  operational 
point  of  view,  while  scores  of  zero  represent  completely  unacceptable 
performance.  But  this  still  gives  no  clues  as  to  what  constitutes  a 
"meaningful"  difference. 

One  standard  against  which  to  evaluate  the  magnitude  of  differences  can  be 
based  on  the  values  obtained  by  pilots  with  different  degrees  and/or  kinds  of 
experience.  Based  on  data  obtained  from  this  experiment  and  other  VTRS 
experiments  (Westra  et  a  1 .  ,  1982),  the  following  rough  estimates  of 
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performance  for  beginning,  intermediate,  and  experienced  levels  of  performance 
can  be  used: 


Beginner 

(Trials 

11-20) 

Intermediate 

(Trials 

49-56) 

Experienced 

Glideslope  TOT 

26 

38 

59 

Lineup  TOT 

49 

68 

N/A 

Angle  of  Attack  TOT 

33 

47 

48 

Wire  Accuracy  Score 

26 

42 

66 

The  differences  between  scores  representing  these  levels  of  ability  would 
generally  be  considered  "moderate"  to  "large"  in  size.  Another  guideline 
suggested  by  Cohen  (1977)  is  to  consider  a  difference  that  accounts  for  a  true 
5  percent  of  the  variance  in  the  population  of  factors  and  subjects  (pilots) 
as  an  effect  of  "moderate"  size.  Based  on  estimates  from  data  obtained  by 
Westra  et  al.  (1982),  differences  that  correspond  roughly  to  this  size  are 
12.0,  10.5,  and  17.0  for  the  glideslope  TOT,  AOA  TOT,  and  wire  accuracy  score, 
respectively.  Effect  sizes  corresponding  to  Cohen's  estimate  of  small  effects 
(1  percent  variance  accounted  for  in  the  population)  are  5.4,  4.7,  and  7.6  for 
the  game  scores. 

GENERAL  DISCUSSION  OF  RESULTS 

The  data  indicate  a  few  substantial,  though  transient,  transfer  effects 
for  the  simulator  factors.  Although  there  were  no  equipment  factor  transfer 
effects  on  touchdown  performance  (Figure  4  and  Table  2),  field  of  view  and 
scene  detail  appear  to  have  temporary  effects  on  final  approach  tracking 
measures.  However,  these  transfer  effects  have  generally  disappeared  after 
eight  transfer  trials. 

The  pilot  type  effect  during  transfer  was  considerably  larger  than 
equipment  factor  effects  (as  it  was  during  training),  even  though  the 
direction  of  the  effect  depended  on  the  dimension  of  performance  measured. 

The  Navy  P-3C  pilots  performed  much  better  on  the  lineup  and  angle  of  attack 
aspects  of  the  final  approach  task,  while  the  Air  Force  T-38  pilots  performed 
better  on  final  approach  glideslope  control  and  landing.  The  reversal  of 
differences  notwithstanding,  this  sing le-degree-of-freedom  representation  of 
pilot  differences  based  on  group  type  generally  accounted  for  more  variance 
than  all  simulator  main  effects  combined.  Despite  the  large  pilot  group 
effects,  pilot  groups  by  equipment  factor  two-way  interactions  were  generally 
small.  This  indicates  that  equipment  factor  effects,  such  as  they  were,  were 
consistent  across  groups. 

Turbulence,  which  was  included  in  the  experiment  primarily  to  allow 
examination  of  the  other  factors'  transfer  effects  under  two  training 
difficulty  levels,  had  little  or  no  transfer  main  effect.  Since  there  does 
not  appear  to  be  strong  evidence  of  a  turbulence  training  effect  either,  the 
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interpretation  of  this  factor  relative  to  its  intended  purpose  is  in 
question.  Therefore,  for  analysis  purposes,  interactions  involving  turbulence 
were  for  the  most  part  considered  "error"  and  treated  as  such. 

DISCUSSION  OF  INDIVIDUAL  FACTORS 

FIELD  OF  VIEW.  Differences  in  the  field  of  view  during  training  had  no  effect 
on  transfer  landing  performance.  There  did  appear  to  be  a  transfer  advantage 
for  the  wide  field  of  view  for  final  approach  measures.  Those  pilots  trained 
on  the  wide  field  of  view  averaged  6.3  percent  more  time  within  .3  degree  of 
the  desired  glideslope.  The  effect  continued  through  the  second  set  of  eight 
transfer  trials  and  accounted  for  8.4  percent  of  the  variance  in  all  transfer 
scores  on  this  measure.  However,  the  effect  is  not  large,  and  significance  at 
the  .01  level  was  not  reached.  In  addition,  an  analysis  (not  reported  nere) 
using  the  root  mean  square  (RMS)  glideslope  error  as  the  dependent  measure 
failed  to  support  the  glideslope  TOT  finding.  As  RMS  error  and  TOT  are 
closely  related,  there  is  considerable  question  about  the  reliability  of  the 
effect.  Further,  since  this  tracking  effect  on  final  approach  glideslope  was 
not  reflected  in  the  touchdown  wire  accuracy  score,  it  probably  should  not  be 
considered  anything  more  than  a  small  effect.  The  field  of  view  did  appear  to 
have  substantial  transient  effects  on  final  approach  lineup  and  angle  of 
attack  transfer  performance. 

These  effects  seem  to  depend  on  approach  type  as  illustrated  in  Figure  8. 
The  transient  lineup  effect  is  associated  with  straight-in  approaches,  while  a 
small  but  persistent  transfer  lineup  advantage  for  the  wide  field  of  view  is 
noted  with  circling  approaches.  For  angle  of  attack  tracking  scores  the 
nature  of  the  transient  effect  depending  on  approach  type  is  reversed.  The 
transient  effect  is  seen  for  circling  trainin  approaches  but  a  persistent 
transfer  advantage  for  the  wide  field  of  view  is  noted  for  straight-in 
traininq  approaches.  It  should  be  noted  that  power  to  examine  individual 
interaction  cell  means  is  weak  in  the  experiment,  and  the  interactions 
themselves  were  not  significant.  Therefore,  results  interpreted  by  approach 
type  should  be  considered  only  suggestive  at  this  point. 

SCENE  DETAIL.  Scene  detail  had  no  transfer  effect  on  the  touchdown  wire 
accuracy  score.  It  did  appear  to  have  some  transient  effects  on  final 
approach  performance.  The  daytime  scene  resulted  in  8.0  percent  more  time 
within  0.3  degree  of  glideslope  for  the  first  block  of  transfer  trials,  but 
this  effect  was  gone  by  the  second  block  of  transfer  trials.  Similar  results 
can  be  seen  for  final  approach  angle  of  attack  and  lineup  tracking  scores  but 
for  the  lineup  measure,  results  depend  on  training  approach  type  as  shown  in 
Figure  9.  There  is  a  sustained  transfer  advantage  for  the  daytime  scene  for 
those  trained  with  straight-in  approaches.  For  those  trained  with  circling 
approaches  there  is  no  transfer  advantage,  temporary  or  otherwise,  for  the 
daytime  scene.  The  effects  on  lineup  and  angle  of  attack  tracking  scores 
should  also  be  considered  only  suggestive  at  this  point. 

MOTION.  Use  of  a  motion  platform  during  training  provided  no  meaningful 
transfer  advantage  over  the  no-motion  condition.  This  was  true  even  though 
the  physical  simulation  of  motion  was  exactly  the  same  in  the  transfer  test  as 
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in  the  training  conditions.  Considering  this  when  attempting  to  generalize 
the  motion  result  to  the  aircraft  leads  to  the  strong  notion  that  simulator 
cockpit  motion  will  provide  no  transfer  benefit.  In  the  real  task  scenario 
motion  cues  must  be  less  than  identical  with  the  simulated  motion  cues  because 
of  the  obvious  physical  limitations  of  a  fixed  based  motion  platform.  Even 
allowing  for  the  possibility  that  subjective  similarity  has  been  or  could  be 
achieved,  there  seems  little  possibility  for  a  positive  motion  transfer  effect 
for  the  carrier  landing  task. 

FLOLS  RATE  CUING.  The  FLOIS  display  with  the  ''command11  rate  information  did 
not  provide  a  transfer  advantage  over  conventional  FLOLS  training  conditions. 
Somewhat  surprisingly,  there  was  no  performance  advantage  for  the  rate  display 
during  training  either.  Previous  research  suggested  that  there  was  a 
considerable  performance  advantage  in  glideslope  control  for  experienced 
pilots  using  FlOLS  displays  with  added  rate  information  (Lintern  et  al.,  1981; 
Kaul  et  al.,  1980).  Apparently,  these  findings  do  not  apply  to  inexperienced 
pilots  who  are  in  the  early  stages  of  learning  the  task. 

APPROACH  TYPE.  Training  with  circling  approaches  provided  no  transfer 
advantage  compared  to  training  with  modified  straight-in  approaches,  even 
though  the  transfer  task  involved  a  circling  approach.  In  fact,  for  the 
lineup  tracking  score,  training  with  straight-in  approaches  resulted  in 
better  transfer  performance  than  training  with  circling  approaches.  This 
effect  was  actually  fairly  large  in  magnitude,  with  straight-in  training 
resulting  in  15.3  percent  more  time  within  1.0  degree  of  the  desired  lineup 
during  final  approach  on  the  transfer  test. 

The  fact  that  training  with  circling  approaches  resulted  in  no  advantage 
on  the  transfer  task  is  probably  in  part  due  to  the  interaction  of  this  factor 
with  task  difficulty.  Approach  type  can  be  thought  of  as  a  training  factor 
with  part-task  and  whole-task  training  levels.  Since  the  task  is  a  very 
difficult  one,  it  might  be  expected  that  the  task  could  be  learned  more 
quickly  with  a  part-task  approach.  The  results  of  this  experiment  generall-y 
support  this  interpretation. 

This  result  could  be  used  to  argue  that  the  field  of  view  issue  is  now 
moot  because  straight-in  approach  training  resulted  in  as  good  or  better 
transfer  than  circling  approach  training.  This  argument  would  be  unjustified, 
however,  for  two  reasons.  First,  Figure  8  indicates  field  of  view  transfer 
differences  on  lineup  and  angle  of  attack  tracking  for  straight-in  approaches 
as  discussed  earlier.  Second,  the  approach  type  matter  is  still  a  training 
issue.  Although  results  from  this  study  support  the  use  of  straight-in 
approaches  during  early  training,  it  is  not  known  whether  circling 
approaches — with  or  without  a  wide  field  of  view — might  be  profitably 
introduced  during  later  training.  (Recall  that  this  experiment  involved 
training  only  to  an  intermediate  level  of  ability.) 

PILOT  TYPE.  As  expected,  differences  between  pilot  types  were  large.  Air 
Force  T-38  pilots  had  better  touchdown  success  and  glideslope  performance  but 
Navy  P-3C  pilots  had  better  lineup  and  angle  of  attack  performance.  However, 
interactions  of  other  factors  with  the  pilot  type  factor  were  generally 
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small.  Thus  effects  were  fairly  consistent  across  groups  despite  the  large 
group  differences,  but  this  is  not  surprising  since  other  effects  were 
general ly  smal 1 . 

Ideally,  results  of  this  experiment  should  be  general izable  to  the 
population  of  interest,  undergraduate  Naval  aviators.  Both  pilot  groups 
represent  pilots  who  have  flying  skills  but  are  learning  a  completely  new 
task.  The  Air  Force  group  is  similar  to  Naval  undergraduates  in  age  and 
experience.  But  a  serious  general i zabi 1 i ty  question  exists  because  both 
groups  brought  learned  skills  to  the  task  that  are  likely  to  result  in 
negative  transfer. 

Specifically,  these  pilots  have  learned  to  control  vertical  glideslope 
deviations  with  elevator  (pitch)  inputs.  The  carrier  landing  task  requires 
vertical  deviation  control  with  throttle  adjustments  while  maintaining  a 
constant  angle  of  attack  with  elevator  inputs.  Acquiring  this  new  control 
strategy  proved  especially  difficult  for  the  Navy  P-3C  pilots,  as  reflected  in 
the  glideslope  tracking  scores.  Their  scores  were  consistently  lower  than  Air 
Force  T-33  pilot  scores  in  this  dimension  of  performance  despite  the  great 
advantage  in  flight  experience.  But  as  there  were  no  strong  indications  from 
the  data  that  results  for  other  experimental  factors  differed  between  the  two 
groups  despite  the  large  group  differences,  it  is  reasonable  to  suggest  that 
results  will  be  similar  for  undergraduate  Naval  aviators. 
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SECTION  IV 

COVARIATE  RESULTS  AND  DISCUSSION 


BACKGROUND 

The  discussion  to  follow  assumes  that  the  reader  has  considerable 
statistical  knowledge.  It  is  of  primary  interest  to  those  concerned  with 
solutions  to  the  problems  of  interpretabi 1 i ty  of  data  from  between  subjects, 
highly  saturated  experiments  such  as  this  one.  The  less  technically-oriented 
reader  is  referred  to  Section  v,  in  which  conclusions  are  presented  that 
adequately  reflect  the  limitations  discussed  in  this  section. 

A  serious  problem  can  exist  when  between  subject  highly  saturated 
multifactor  experimental  designs  employing  one  (or  few)  subjects  per  condition 
are  used.  The  transfer  of  training  design  used  in  the  experiment  reported 
here,  in  which  there  was  one  subject  in  each  of  32  experimental  conditions  is 
an  example  of  such  a  design.  In  such  a  design  every  estimable  effect  is 
confounded  with  differences  due  to  subjects  represented  by  the  particular 
groups  that  define  a  specific  comparison.  For  example,  the  estimate  of  the 
motion  effect  in  this  experiment  is  composed  of  a  true  motion  effect  plus  the 
true  mean  difference  in  level  of  performance  between  the  16  pilots  in  the 
no-motion  and  the  16  pilots  in  the  motion  conditions,  plus  other  error.  Thus 
the  subject  difference  effect  biases  the  estimate  of  the  factor  effect,  and  by 
an  unknown  amount. 

Consideration  of  this  problem  requires  some  definition  of  terms  and 
description  of  conditions  that  underlie  the  area  of  concern.  First,  it  is 
assumed  that  large,  predictable  true  differences  between  subjects  exist  in 
terms  of  their  level  of  ability  on  the  task  at  hand.  (True  subject 
differences  in  rate  of  learning  ability  also  enter  into  the  problem  for 
transfer  of  training  designs,  but  this  will  not  be  considered  for  the 
moment.)  To  the  extent  that  subject  differences  do  not  exist  or  are  very 
small  relative  to  other  factor  effects,  the  problem  does  not  exist. 

If  subject  differences  exist  but  these  differences  are  known  or  can  be 
estimated,  the  biasing  problem  is  effectively  eliminated.  Thus,  for  example, 
a  within  jbjects  design  does  not  have  this  confounding  problem  because 
subject  oi  references  can  be  estimated  and  "removed"  from  other  sources  of 
variance.  Obviously,  if  a  "perfect"  covariate  existed  for  a  between  subject 
design,  the  problem  would  also  be  eliminated.  The  identification  of  subjects' 
mean  level  of  ability  then  represents  a  solution  to  the  problem. 

The  term  "highly  saturated"  here  refers  to  a  design  in  which  there  is  an  a 
priori  interest  in  a  large  oercentage  of  the  possible  orthogonal  estimable 
effects.  A  typical  few  fac.or  between  subjec.  design  with  more  than  a  few 
subjects  per  condition  is  not  saturated  because  only  a  few  of  the  possible 
effect  estimates  in  the  orthogonal  set  def’ieu  by  the  design  are  examined. 

The  rest  are  combined  as  part  of  some  "err  •-  estimate.  Another  way  to  think 
of  this  is  that  in  the  fully  saturated  one-'-,  iect-per-condition  design,  the 
pool  of  subjects  is  split  into  all  possible  groupings  of  an  orthogonal  set. 
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Then  each  grouping  is  associateu  witn  an  effect  anu  group  differences  are 
confounded  with  the  effects  with  whicn  they  are  associated. 

This  can  be  seen  by  referring  to  the  eignt  condition  design  presented  in 
Table  6  whicn  shows  eacn  effect  associated  with  one  of  its  seven  orthogonal 
groupings  of  subjects.  (The  particular  groupings  follow  from  an  original 
random  assignment  of  subjects  to  conditions.)  In  an  unsaturated  one-factor 
design  with  the  same  number  of  subjects,  there  would  be  an  interest  in  only 
one  of  the  subject  groupings  and  its  associated  effect,  say  0  in  Table  6.  All 
the  other  effects  would  be  used  to  estimate  an  error  term.  Clearly,  the  bias 
problem  also  exists  to  some  degree  in  a  one-factor  design  with  many  subjects 
per  condition.  There  is  a  oossibility  of  getting  all  the  "best"  subjects  in 
one  group  and  all  the  "worst"  subjects  in  another.  Investigators  generally 
trust  that  random  assignment  ana  the  laws  of  probability  will  usually  save 
them  from  such  an  outcome.  If  a  more  even  split  of  subjects  occurs,  with 
roughly  equal  numbers  of  good  and  bad  subjects  in  eacn  group,  the  amount  of 
bias  will  be  small  and  probably  trivial. 


TABLE  6. 

FULLY 

SATURATED  i 

BE  TWEEN 

SUBJECTS 

DtSluN* 

Effect 

Cond i tion 

Subject 

A 

B 

C 

!) 

E 

F  G 

Mean 

1 

1 

_ 

4- 

__ 

4- 

4-  _ 

4- 

2 

2 

+ 

_ 

- 

— 

— 

+  4* 

+ 

3 

3 

- 

4- 

- 

- 

+ 

_  4- 

4* 

4 

4 

+ 

4- 

+ 

_ 

_ 

__  __ 

4* 

5 

5 

- 

- 

4- 

+ 

- 

_  4- 

4* 

6 

6 

4- 

- 

- 

+ 

4- 

- 

4- 

7 

7 

+ 

_ 

+ 

- 

4-  _ 

+ 

8 

8 

+ 

+ 

4- 

4- 

+ 

4-  4- 

4* 

*A  1  1 

factors  at  two 

1 evel s 

ii  +  ii 

and 

distinguish 

levels  or 

the  factors 

In  fact,  given  a  sufficient  number  of  subjects,  the  probability  of  getting 
a  very  bad  sampling  break  is  small.  For  example,  in  a  one-factor,  two-level 
experiment  with  32  subjects  (16  per  qroup),  and  assuming  the  subjects  can  be 
classified  into  16  "good"  and  16  "baa"  performers  based  on  mean  level  of 
ability,  the  probability  of  getting  the  worst  sampling  break  is  =  3.0  x 
10-9.  the  probability  of  getting  the  next  worst  sampling  break  is  8.5  x 
lO-^  ancj  the  probability  for  next  worst  is  4.8  x  10“^.  On  the  other  hand, 
probabilities  for  getting  an  even  break  (8  good,  8  bad  subjects  in  each  group) 
or  a  nearly  even  break  (9  good  and  7  bad  in  one  group  and  9  bad  and  7  good  in 
the  other)  are  .276  and  .435,  respectively. 


These  same  probabilities  hold  for  an  a  priori  consideration  of  one  of  the 
effects  in  a  saturated  one-subject-per-condi tion  multifactor  design.  However, 
by  implication  of  the  design  there  is  an  intention  to  examine  many  of  the 
effects.  Because  all  of  the  possible  orthogonal  subject  qroupi'gs  are 


36 


N AVTRAEOU 1 PCEN  31-C-0105-1 


involved  in  these  effects,  ana  tnas  all  of  the  subject  variability  is  involved 
in  the  confounding,  it  is  virtually  certain  that  some  effects  will  involve 
"baa"  subject  sampling  breads.  The  hign  probability  of  getting  at  least  a  "ew 
heavily  biased  effects  defines  tne  crux  or  the  problem. 

Without  knowledge  of  subjects'  mean  levels  of  ability,  there  is  no  way  of 
knowing  whicn  effect  estimates  are  neavily  biased  by  subject  differences. 

Since  it  is  virtually  certain  that  some  estimates  will  be  heavily  biaseo,  one 
would  be  faced  with  uni nterpretabl e  data  in  such  a  situation.  An  effect 
estimate  could  be  large  totally  due  to  subject  differences  or  an  effect 
estimate  could  be  small  because  a  subject  difference  is  in  the  opposite 
direction  of  the  experimental  effect  with  whicn  it  is  confounded.  The  problem 
will  always  be  serious  when  dealing  with  complicated  motor  control  tasks  for 
which  subject  differences  typically  account  for  20  percent  or  more  of 
performance  variance. 

It  snould  be  noted  that  tne  problem  coula  also  be  partly  described  as  one 
of  multiple  comparisons  from  the  same  set  of  data.  In  this  sense,  within 
subject  variability  for  a  witmn  subjects  design  represents  the  same  kind  of 
problem.  However,  within  subjects  variability  is  generally  assumed  small 
relative  to  between  subject  variability,  and  it  is  typically  much  easier  ana 
cheaper  to  obtain  data  on  many  runs  from  the  same  subject  than  it  is  to  obtain 
many  subjects.  Thus,  the  within  subject  source  of  error  problem  is  considerea 
categorically  different  from  a  practical  point  of  view. 

Because  of  the  severity  of  the  problem,  it  is  mandatory  that  research  on 
identification  of  subjects'  level  of  ability  be  an  integral  part  of  any 
research  program  in  which  highly  saturated  between  subject  desiqns  will  be 
used.  Further,  basic  research  on  the  appropriate  use  of  knowledge  of  between 
subject  differences  with  these  designs  remains  to  be  done.  The  problem  is  not 
insurmountable  and  there  are  several  possibilities  for  reducing  the  degree  of 
risk  wnile  sacrificing  little  of  the  great  economy  these  designs  have  to 
offer.  It  snould  also  be  pointed  out  that  this  problem  does  not  mitigate  the 
economy  of  the  holistic  approach.  For  a  given  degree  of  subject  variability, 
the  arguments  in  favor  of  a  multifactor  approach  remain  the  same.  However, 
thorough  exposition  and  statistical  treatment  of  tne  problem  remains. 

EXPERIMENTAL  SUBJECT  VARIABILITY 

Probably  most  significant  in  terms  of  controlling  subject  variability  in 
this  experiment  was  the  inclusion  of  pilot  type  (Air  Force  T-38  vs.  Navy  P-3C) 
directly  as  an  experimental  factor.  This  type  of  subject  variability  control 
is  best  of  all,  of  course,  because  the  guesswork  due  to  less  than  perfect 
covariates  is  done  away  with.  Differences  on  the  defined  dimensions  of 
subjects  are  estimated  directly  as  part  of  the  experiment.  This  subject 
factor  almost  completely  accounts  for  flight  experience  differences  also, 
accounting  for  77  percent  of  the  variance  among  pilots  described  by  number  of 
flight  hours.  This  is,  of  course,  primarily  due  to  the  fact  that  all  Air 
Force  pilots  had  little  flight  experience  compared  to  the  Navy  pilots. 

Because  flight  experience  is  a  strong  covariate  describing  subject  level  of 
ability  (for  example,  see  Westra  et  al.,  1982),  there  was  reason  to  believe 
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that  the  pilot  type  factor  would  account  for  a  large  percentage  of  total 
predictable  subject  variability. 

Tnis  is  an  important  considerat ion  bearing  on  the  proDlem  of  subject 
effect  biasing  described  earlier.  If  an  effect  represents  a  "best"  vs. 

"worst"  subjects  comparison,  it  will  account  for  most  of  the  identifiable 
variance  due  to  subjects  in  the  entire  experiment,  depending  on  the 
distribution  of  subject  levels  of  ability.  For  example,  if  the  two  groups 
represent  homogeneous  groups  differing  only  in  mean  level  of  ability,  this 
single  effect  will  account  for  essentially  all  of  the  predictable  subject 
differences.  If  subject  mean  levels  of  ability  are  ordereo  with  equal 
intervals  from  worst  to  best,  the  "best"  vs.  "worst"  contrast  accounts  for  75 
percent  of  the  identifiable  subject  differences  (Simon,  1977b).  In  this  case, 
two  other  effects  can  be  identified  which  account  for  19  percent  ana  5  percent 
of  the  remaining  variability  due  to  subjects.  All  other  effects  will  have 
less  tnan  1  percent  overlap  with  subject  differences. 

Thus,  if  only  one  effect  can  be  identified  and  associated  with  the  largest 
single  source  of  variability  due  to  subjects,  most  of  the  remaining  effect 
estimates  are  likely  to  be  only  trivially  biasea  by  subject  differences. 
Further,  given  enough  information  to  oruer  subjects  on  tneir  mean  levels  of 
anility,  23  effect  estimates  in  a  32-condition  design  can  be  obtained  wnicn 
are  essentially  free  of  biases  due  to  subject  differences. 

In  the  present  study,  there  is  considerable  justification,  Dotn  on  a 
priori  grounds  and  on  oost  hoc  grounds  from  an  examination  of  effect  sizes 
associated  with  the  pilot  type  factor,  for  believing  that  the  pilot  type 
factor  accounted  for  a  large  proportion  of  the  variance  due  to  subjects.  To 
the  extent  that  this  is  so,  it  is  reasonable  to  assume  that  most  of  the  other 
estimable  effects  in  the  experiment  were  only  trivially  biased  by  subject 
differences. 

ATARI  AIR  COMBAT  MANEUVERING  RESULTS 

For  a  covariate  to  be  effective,  several  requirements  must  be  met 
simultaneously.  First,  the  measures  on  the  task  must  have  high  test-retest 
reliability  which  implies  stronq  capability  to  discriminate  subject  levels  of 
ability.  The  reliability  should  remain  high  for  a  sufficient  period  of  time 
over  continued  practice  on  the  covariate  task  which  implies  stability  of 
subject  relative  levels  of  performance.  This  is  especially  important  when 
considering  psychomotor  tasks  for  which  considerable  learning  will  occur. 
Because  subjects  learn  at  different  rates,  stability  may  not  occur  until  after 
considerable  practice.  Second,  performance  on  the  criterion  task  must  have 
high  test-retest  reliability.  The  stability  requirement  isnot  as  critical 
here  since  the  goal  is  to  predict  subject  levels  of  ability  at  the  time  of  the 
experiment  (or  transfer  in  a  transfer  of  training  experiment).  Still,  without 
some  reasonable  stability  at  the  time  the  key  data  are  collected,  the 
predictive  capability  of  the  covariate  will  probably  be  compromised. 

Third,  the  relationship  between  the  covariate  task  and  criterion  task  must 
be  strong.  This  strength  of  association  is  practically  limited  by  the 
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reliability  of  the  measures  on  covariate  and  criterion  tasks.  If  the  score  on 
either  has  low  reliability,  there  is  no  possibility  for  accurately  estimating 
the  subject's  mean  levels  of  ability  from  the  data.  Note  tnat  the  ultimate 
goal  of  having  the  highest  possible  true  relationship  between  covariate  and 
criterion  task  performance  at  the  desired  time  does  not  absolutely  require 
stability  of  relative  subject  performance  levels  on  the  covariate  task.  Again 
though,  unless  a  reasonable  degree  of  stability  exists,  the  practical  strength 
of  the  relationship  between  covariate  and  criterion  will  probably  be 
compromised.  In  any  case,  ?.  prerequisite  to  the  use  of  covariate  data  in  the 
analysis  of  experimental  results  from  saturated  designs  is  an  examination  of 
the  within  and  between  task  relationships. 

The  first  16  pilots  in  the  present  study  performed  50  ACM  simulator 
trials.  Examination  of  the  correlation  matrix  for  these  pilots*  scores  on 
five-trial  means  indicated  stabilized  performance  essentially  from  the  start. 
These  correlations  are  shown  in  Table  7. 


TABLE 

7.  INTERCORRELATIONS  FOR  FIVE 

-TRIAL 

MEANS 

FOR  FIFTY 

TRIALS  ON 

AIR  COMBAT  MANEUVERING 

(16  PILOTS) 

Mean  of 
Tri als 

1-5 

6-10 

11-15 

16-20 

21-25 

26-30 

31-35 

36-40  41-45 

6-10 

.81 

11-15 

.83 

.83 

16-20 

.90 

.81 

.88 

21-25 

.81 

.87 

.82 

.90 

26-30 

.81 

.79 

.79 

.89 

.84 

31-35 

.89 

.90 

.87 

.89 

.89 

.87 

36-40 

.88 

.80 

.76 

.86 

.81 

.74 

.83 

41-45 

.73 

.76 

.80 

.81 

.81 

.73 

.85 

.84 

46-50 

.89 

.87 

.81 

.85 

.87 

.81 

.91 

.85  .89 

As  this  finding  indicated  that  a  small  number  of  trials  was  required  for 
stability,  the  remaining  pilots  were  tested  on  30  ACM  trials.  In  subsequent 
analyses,  each  pilot's  mean  score  for  trials  26-30  was  considered  to  be  the 
covariate.  The  correlation  matrix  for  all  pilots'  scores  on  five-trial  means 
is  given  in  Table  8.  Clearly,  the  ACM  task  meets  the  requirements  of 
stability  and  high  reliability.  The  average  correlation  between  blocks  of 
five  trials  after  the  third  block  was  *  .85. 


NAVTRAEQU I PCEN  81-C-0105-1 


TABLE  8. 


INTERCORRELATIONS  FOR  FIVE-TRIAL  MEANS  FOR  THIRTY 
TRIALS  ON  AIR  COMBAT  MANEUVERING  (32  PILOTS) 


Mean  of 


Tr i al s 

1-5 

6-10 

11-15 

16-20 

21-25 

6-10 

.81 

11-15 

.76 

.77 

16-20 

.89 

.80 

.83 

21-25 

.79 

.77 

.79 

.89 

26-30 

.76 

.65 

.74 

.87 

.86 

Scores  on  the  last 
0.3  degree  on  glideslope 
accuracy  score  were  used 
performance.  Only  this 
reasonably  "safe"  to  use 
However,  it  must  De  kept 
"safe"  these 
confounding . 


block  of  eight  transfer  trials  for  percent  time  within 
(3000  feet  to  the  ramp),  and  the  touchdown  wire 
to  examine  the  relationship  between  ACM  and  simulator 
last  block  of  eight  transfer  trials  was  considered 
in  terms  of  independence  from  transfer  effects, 
in  mind  that  it  is  not  possible  to  determine  how 
transfer  trials  are  because  of  possible  subject  effect 
Further,  since  there  was  a  known  pilot  type  effect  througn  these 


transfer  trials,  the  relationship  within  pilot  type  was  also  examinea.  The 
relative  sparsity  of  usable  information  on  the  criterion  task  for  covariate 
validation  purposes  does,  of  course,  restrict  the  general izabi 1 ity  of  the 
f i nd i ng  s . 


The  correlations  between  mean  scores  on  blocks  of  four  of  the  last  eight 
transfer  trials  are  shown  in  Table  9,  along  with  the  correlations  between  ACM 
and  glideslope  and  landing  scores  on  the  last  eight  transfer  trials.  As  the 
table  indicates,  reliabilities  for  touchdown  scores  and  glideslope  tracking 
scores  are  poor.  This  could  be  in  part  due  to  the  low  number  of  trials  used 
to  obtain  a  pilot's  mean  score.  In  contrast,  the  high  reliabilities  given  by 
Xestra  et  al.  (1982),  were  based  on  means  from  blocks  (experiments,  actually) 
of  32  trials  from  experienced  pilots.  The  low  reliabilities  could  also  be  due 
in  part  to  lack  of  stabilization  on  the  task.  There  was  evidence  suggesting 
poor  stabilization  for  the  Navy  group  in  particular,  at  this  point  in 
training,  even  though  it  was  the  end  of  the  training  experiment.  This 
sugqests  the  training  period  miqht  have  been  too  short,  among  other  thiriqs. 
Given  the  low  reliabilities  for  scores  on  the  criterion  task,  the 
relationships  with  the  ACM  covariate  are  generally  as  strong  as  could  be 
expected. 

EFFECT  OF  ACM  AS  A  COVARIATE  ON  EXPERIMENTAL  RESULTS 


Rather  than  present  an  analysis  of  covariance  with  the  available  degree  of 
justification  for  use  of  ACM  as  a  covariate,  the  relationship  of  the  covariate 
to  the  estimable  terms  in  the  design  will  be  examined.  Effect  estimates  from 
terms  that  share  little  or  no  variance  with  the  covariate  would  not  be 
affected  by  a  covariate  adjustment,  i.e.,  these  terms  are  "robust"  to  subject 
effects  just  like  most  terms  in  a  multifactor  design  are  "robust"  to  trend 
effects  (Simon,  1977b).  Examination  of  correlations  between  covariate  and 
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TABLE  9.  CORRELATIONS  OF 
CARRIER-LANDING 

Air  Force  T-38 
(N-16) 


Glideslopel  by 

ACM  Correlation  .37 

Landing^  by  ACM 
Correlation  .36 

G1  ideslope 

Reliability^  .32 

Landing  Reliability  .29 


STABLE  ACM  SCORES  WITH  SIMULATED 
TRANSFER  TASK  SCORES 


Navy  P-3C 

A1 1  Pi  lots 

( N  =1 6 ) 

( N =3  2 ) 

.25 

.42* 

.48* 

.51* 

.23 

.32 

.20 

.35* 

1-Mean  glideslope  tracking  scores  for  last  eignt  transfer  trials.  Score  was 
percent  time  within  0.3  degree  of  desired  for  3000  ft.  to  ramp. 

^Mean  touchdown  wire  accuracy  score  for  last  eight  transfer  trials. 

^Re 1 i abi 1 ities  are  the  correlations  of  the  last  four  transfer  trial  means 
for  each  pilot  vs.  the  preceding  four  transfer  trial  means. 

*  p  <  .05 


estimable  terms  identifies  those  terms  associated  with  subject  effects  based 
on  the  covariate.  Theoretically,  assuming  a  covariate  that  perfectly  predicts 
subject  differences  on  the  criterion  task,  this  is  all  that  it  is  necessary  to 
know  to  design  multifactor  few-subject s-per-cel 1  experiments.  The  terms 
correlated  with  the  covariate  would  be  left  with  no  factor  assignments  and 
simply  called  subject  effects  in  the  analyses.  These  terms  would  then  do  the 
work  of  removing  subject  differences  from  the  results. 

The  correlations  between  ACM  means  on  trials  26-30  and  the  31  estimable 
terms  in  the  experimental  design  are  shown  in  Table  10.  Only  three  individual 
terms  are  associated  with  more  than  6  percent  of  the  total  variance  described 
by  ACM  as  a  covariate.  These  three  effect  estimates  are  the  only  ones  that 
would  be  more  than  trivially  affected  by  a  covariate  adjustment.  In  other 
words,  under  the  assumption  that  ACM  is  a  perfect  descriptor  of  subject  mean 
level  of  ability  on  the  carrier  landing  task,  all  experimental  effect 
estimates  are  essentially  unbiased  by  subject  differences  (although  there 
might  be  some  debate  about  the  triviality  of  6  percent  variance  overlap) 
except  for  the  three  indicated  terms. 

One  of  the  three  terms  associated  with  ACM  is  the  pilot  type  factor  with 
14  percent  variance  overlap  indicated.  Since  this  term  was  already  defined  as 
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a  subject  effect,  the  effect  estimate  simply  estimated  this  amount  of  subject 
difference.  The  fact  that  this  particular  term  was  defined  as  a  subject 
effect  is,  of  course,  exactly  what  is  desired.  Other  than  this,  what  is 
noteworthy  here  is  that  ACM  correctly  predicts  that  Air  Force  pilots  will  have 
better  glideslope  tracking  scores  than  Navy  P-3C  pilots,  exactly  opposite  the 
effect  direction  predicted  by  flight  experience.  This  lends  additional 
support  to  the  use  of  ACM  as  a  covariate  for  this  task. 

The  other  two  terms  having  more  than  trivial  overlap  with  the  covariate 
were  defined  as  the  motion  factor  and  a  three-way  interaction  string.  The 
positive  correlation  of  motion  with  ACM  suggests  that  more  of  the  "better" 
(based  on  the  covariate)  pilots  were  in  the  motion  condition.  Results  for 
glideslope  tracking  and  landing  performance  (Figures  4  and  5;  Tables  2  and  3), 
without  covariate  adjustment,  show  a  very  small  positive  transfer  benefit  for 
motion  training  which  cannot  be  differentiated  from  noise.  The  effect  of  a 

TABLE  10.  CORRELATIONS  OF  STABLE  ACM  SCORES  WITH  ESTIMABLE 
TERMS  IN  THE  EXPERIMENTAL  DESIGN 


Correlation 

Percent 

Source  of  Variance 

with  ACM 

Over  1 ap 

Field  of  View  (A) 

.00 

0 

Motion  (B) 

.35 

12 

Pi  lot  Type  (C) 

-.37 

14 

Scene  Detai 1  (0 ) 

-.21 

4 

Turbulence  (E) 

-.05 

0 

FLOLS  Rate  Cuing  (F) 

-.18 

3 

Approach  Type  (G) 

-.09 

1 

BD  +  AF 

.02 

0 

AB  +  OF 

.12 

1 

BC 

-.18 

3 

AD  +  BF 

-.08 

1 

CD 

-.03 

0 

OE 

-.19 

4 

AC 

.02 

0 

AE 

.03 

0 

CE 

-.11 

1 

8 DC  +  ACF 

-.05 

0 

BED  +  AEF 

.01 

0 

EG 

-.25 

6 

CG 

.21 

1 

ADC  ♦  BCF 

-.20 

4 

AED  +  BEF 

.21 

4 

FG 

.00 

0 

BG 

.06 

0 

CF 

.22 

5 

EF 

.07 

0 

ADF  +  BFG 

-.46 

21 

BDG  +  AFG 

.24 

6 

DG 

-.11 

1 
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covariate,  adjustment  then  would  be  to  decrease  the  size  of  the  effect  and 
perhaps  even  change  the  direction  of  the  effect  to  a  transfer  advantage  for 
the  no-motion  training  conditions.  In  fact,  in  some  covariance  analyses  that 
were  run,  the  motion  effect  was  reduced  in  magnitude  but  did  not  change 
direction.  Since  the  motion  effect  was  not  significant  to  begin  with,  the 
impact  of  this  is  simply  to  emphasize  the  bottom  line,  no  differential  motion 
transfer  effect. 

The  three-way  interaction  string  shows  the  strongest  individual 
relationship  with  the  covariate  (21  percent  overlap).  This  effect  was  always 
treated  as  ''error'1  in  the  reported  analyses  and  thus  was  part  of  the  term  used 
as  an  estimate  of  the  "subject  within  groups"  variance  described  in  classical 
tests  (Winer,  1971,  ch.7).  Assuming  that  ACM  is  an  appropriate  covariate,  the 
result  of  use  of  the  effect  in  this  way  then  was  simply  to  make  the  F-tests 
more  conservative  than  they  would  have  been  had  this  effect  been  labeled  a 
subject  effect  and  not  included  in  the  between  subjects  term. 
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SECTION  V 

SUMMARY  ANO  CONCLUSIONS 

An  in-simulator  transfer  of  training  experiment  was  conducted  with  novice 
carrier-landing  pilots  as  part  of  an  effort  to  define  design  requirements  for 
undergraduate  carrier-1 anding  simulators.  Three  simulator  equipment  factors 
were  investigated  (field  of  view,  scene  detail,  and  cockpit  motion)  along  with 
two  training  factors  (approach  type  and  FLOLS  rate  cuing)  and  one 
environmental  factor  (turbulence).  A  pilot  group  factor  defined  by  two 
populations  with  no  carrier-landing  experience  (Air  Force  T-38  and  Navy  P-3C 
pilots)  was  also  included  in  the  experiment.  Pilots  performed  AO  training 
trials  with  instructional  feedback  under  conditions  defined  by  various  levels 
of  the  experimental  factors  and  then  performed  16  transfer  test  trials  under  a 
high-fidelity  simulator  configuration. 

The  main  conclusion  to  be  drawn  from  the  results  is  that  transfer  effects 
due  to  equipment  variables  were  fairly  small  from  a  practical  point  of  view. 
Transfer  landing  quality  was  not  affected  by  any  factor  other  than  pilot  type 
and  approach  quality  was  generally  only  temporarily  affected  by  equipment 
factors,  i.e.,  effects  had  essentially  disappeared  after  eight  transfer 
trials.  The  pilot  type  effect  was  generally  larger  than  all  other  effects 
combined  and  the  only  other  factor  that  had  a  sizable,  sustained  transfer 
effect  was  a  training  factor,  approach  type. 

The  temporary  effects  that  did  occur,  however,  were  not  insubstantial. 

The  wide  field  of  view  and  high  scene  detail  conditions  generally  resulted  in 
better  final  approach  performance  during  the  first  eight  transfer  trials. 

This  suggests  that,  for  an  undergraduate  carrier  landing  trainer,  some 
training  advantage  could  result  from  the  use  of  a  wide-angle  high-detail 
visual  system  over  the  narrow-angle  lower-detail  alternative.  Whether  this 
advantage  would  result  in  cost  savings  sufficient  to  offset  the  added 
simulator  procurement  and  life-cycle  costs  is  questionable  due  to  the  short 
duration  of  the  effect.  Nevertheless  the  results  are  sufficiently  suggestive 
to  justify  a  simul ator-to-f ield  transfer  of  training  experiment  in  which  the 
variables  of  field  of  view,  scene  detail  and  approach  type  are  manipulated. 

This  experiment,  combined  with  those  of  Westra  et  al.  (1982),  have  thus 
provided  a  strong  basis  for  deciding  which  variables  are  most  important  for 
inclusion  in  a  much  more  costly  and  difficult  field  transfer  experiment.  The 
other  factors  examined  in  these  experiments  are  judged  far  less  likely  to  be 
of  operational  significance  and  are  not  recommended  for  inclusion  in  such  a 
field  study. 

INDI V IDUAL  FACTORS 

A  brief  summary  of  results  follows  with  the  factor  effects  listed  in  the 
order  of  overall  impact  on  the  transfer  task. 
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Pilot  Type  had  by  far  the  largest  effect  during  the  transfer  trials  but 
the  effect  was  dependent  on  the  dimension  of  performance  measured.  Air  Force 
T-38  pilots  did  better  than  Navy  P-3C  pilots  on  glideslope  tracking  and 
landing  wire  accuracy,  but  Navy  P-3C  pilots  did  much  better  at  lineup  tracking 
and  angle  of  attack  control.  The  straight-in  approach  type  training  resulted 
in  better  final  approach  lineup  control  on  the  transfer  task  despite  the  fact 
that  the  transfer  task  involved  a  circling  approach.  Pilots  training  with 
straight-in  approaches  had  15  percent  more  time  within  the  lineup  tolerance 
limit  on  the  transfer  task  than  pilots  training  with  circling  approaches. 

Also,  the  circling  approach  training  showed  no  transfer  advantage  for  other 
final  approach  scores  or  landing  accuracy. 

The  wide  field  of  view  resulted  in  some  advantage  on  the  transfer  task  for 
final  approach  quality  but  not  landing  accuracy.  There  was  a  small  effect  on 
glideslope  tracking  and  a  temporary  effect  on  lineup  and  angle  of  attack 
tracking.  The  overall  effect  on  final  approach  quality  after  eight  transfer 
trials  was  small  at  best. 

Daytime  scene  detai 1  training  conditions  had  no  transfer  advantage  over 
night  scene  detail  training  conditions  on  landing  accuracy.  There  were 
transient  effects  on  final  approach  glideslope  and  angle  of  attack  performance 
favoring  the  daytime  training  scenes.  There  appeared  to  be  a  final  approach 
lineup  performance  advantage  on  transfer  with  the  daytime  training  scenes  but 
only  with  straight-in  training  approaches. 

The  presence  or  absence  of  motion  during  training  did  not  make  a 
difference  on  transfer.  The  addition  of  rate  cuing  information  to  the  FLOlS 
did  not  result  in  a  transfer  advantage  compared  to  training  with  a 
conventional  FlOlS. 

CONSIDERATIONS 

Several  points  should  be  kept  in  mind  when  considering  these  results. 
First,  the  serious  problem  of  subject  effect  confounding  was  discussed  in 
detail.  Although  a  subject  factor  was  included  in  the  experiment  that 
appeared  to  account  for  a  large  amount  of  subject  variance,  and  a  covariate 
task  was  employed  that  apparently  showed  some  relationship  to  the  criterion 
task,  this  problem  was  not  fully  resolved. 

It  should  also  be  noted  that  in  keeping  with  the  "screening"  nature  of  the 
design,  factor  levels  chosen  for  study  represent  extremes  of  the  operationally 
reasonable  range  of  interest.  It  is  not  suggested  that  "low"  fidelity  be 
strictly  defined  by  the  low-level  conditions  of  the  experiment.  Improvements 
to  low-level  conditions  that  can  be  made  at  little  or  no  cost  should  always  be 
considered,  particularly  if  there  is  some  justification  from  experimental 
data.  For  example,  both  field  of  view  and  scene  detail  had  moderately  large 
effects  on  roll  variability  during  the  training  trials  with  the  high-fidelity 
versions  of  these  factors  resulting  in  reduced  roll  variability.  While  these 
effects  may  not  have  been  large  enough  to  affect  task  outcome  substantially, 
they  do  suggest  that  the  "flyability"  of  the  simulator  in  terms  of  workload  is 
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affected  by  these  factors.  The  effect  on  roll  variability  is  almost  certainly 
a  function  of  the  extent  and  visibility  of  the  horizon,  varying  from  a  full 
horizon  with  the  wide  field  of  view  daytime  high-detail  scene  to  none  at  all 
with  low-detail  scenes.  This  suggests  that  a  low  cost  horizon  for  the 
low-detail  scene  should  be  considered  even  if  task  outcome  transfer  results 
show  no  difference. 

Finally,  there  is  a  question  of  general izabi 1 ity  o'  results  because  of 
possible  negative  transfer  of  skills  from  the  subject  pilot  populations  to  the 
task.  A  s imul ator-to-f ield  study  with  undergraduate  Naval  aviators  is  needed 
to  confirm  the  results  obtained  here.  This  study  provides  recommendations  for 
such  an  experiment  and  at  the  same  time  depends  on  a  field-transfer  study  for 
its  own  ultimate  value.  Confirmatory  results  could  go  a  long  way  toward 
increasing  confidence  in  in-simulator  results  and  saving  some  of  the  enormous 
expense  associated  with  field-transfer  studies. 
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APPENDIX  A 
BRIEFING 

CARRIER  LANDINGS  IN  THE 
VISUAL  TECHNOLOGY  RESEARCH  SIMULATOR 


INTRODUCTION 

Welcome  to  the  Visual  Technology  Research  Simulator  (VTRS).  This  is  a 
Naval  Research  Facility  developed  to  study  the  use  of  simulators  for  teaching 
flight  skills. 

The  VTRS  simulates  a  T-2C  aircraft  and  consists  of  a  single  seat  cockpit, 
a  ten  foot  radius  spherical  screen  which  surrounds  the  cockpit,  and  control 
computers  which  run  the  simulator.  The  cockpit  controls  and  instruments 
operate  just  as  they  do  in  a  real  aircraft.  A  picture  of  an  aircraft  carrier 
is  projected  on  the  screen,  and  when  the  simulator  is  running,  the  scene  will 
look  just  as  it  would  if  you  were  flying  a  real  carrier  approach. 

Because  this  is  a  controlled  experiment,  we  will  be  using  a  special 
sequence  and  schedule  to  instruct  you  in  what  you  are  to  learn.  This  is  to 
assure  that  each  person  in  the  experiment  receives  the  same  material  in 
exactly  the  same  manner.  However  be  sure  to  ask  for  clarification  on  any 
points  you  do  not  understand. 

We  are  teaching  different  people  under  different  conditions.  While  we  do 
not  believe  that  knowledge  about  other  conditions  will  affect  your 
performance,  we  would  like  you  to  inhibit  your  curiosity  about  what  others  are 
doing  until  your  experimental  work  is  over.  It  is  possible  that  viewing  the 
displays  at  the  control  station  could  affect  your  performance,  so  we  would 
like  you  to  wait  in  the  subject  room  if  you  arrive  early  for  a  session.  Brief 
exposure  to  the  control  station  displays  or  those  found  elsewhere  in  the  VTRS 
building  will  not  affect  you,  but  please  do  not  spend  any  substantial  amount 
of  time  studying  them. 

We  will  tell  you  when  you  have  finished  the  experiment  and  will  be 
prepared  to  describe  other  conditions  at  that  time,  or  to  let  you  view  the 
control  station  operation  if  you  wish. 

We  appreciate  your  participation  in  this  experiment  and  we  hope  that  it 
will  be  a  meaningful  experience  for  you. 

ABOUT  THE  EXPERIMENT 


This  study  has  been  designed  to  tell  us  something  about  how  simulation  can 
be  used  to  teach  carrier  landings.  We  are  examining  the  training  efficiency 
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of  several  different  instructional  techniques  by  teaching  carrier  landings 
under  the  different  configurations  and  then  testing  landing  ability  in  a 
simulator  configuration  that  is  as  close  to  full  fidelity  as  we  can  get. 

While  the  experiment  will  be  conducted  entirely  in  the  simulator  *,e  1  itend  to 
use  the  information  gathered  from  it  to  help  us  design  a  study  in  which  pilots 
will  be  taught  first  in  the  simulator  and  then  tested  in  the  aircraft. 

Mote  that  this  experiment  is  aimed  at  testing  the  simulator,  and  is  not  a 
test  of  your  ability.  Nevertheless  there  will  be  differences  between  pilots 
and  we  need  to  account  for  these  when  we  analyze  the  data.  Differences  will 
be  minimized  if  everyone  does  their  best.  We  would  like  you  to  concentrate  on 
learning  the  task  in  tne  correct  manner  and  as  quickly  as  possible.  We  would 
also  like  you  to  do  your  best  on  every  trial. 

For  a  normal  day  carrier  landing  tne  pilot  circles  the  carrier  to  fly  a 
downwind  leg  parallel  to  and  in  the  opposite  direction  to  the  carrier  neading, 
and  about  one  mile  to  port  (left  looking  towards  the  bow).  This  is  where  you 
will  start  your  circling  approaches.  The  simulator  will  be  flying  straight 
and  level  in  the  landing  configuration  (wheels  down,  flaps  down,  hook  down  and 
brakes  out)  with  15  units  Angle  of  Attack  ( AOA) ,  600  feet  of  altitude,  35 
percent  power,  and  on  a  heading  of  180°.  Start  a  left  turn  with  15°  to  1S°  of 
bank  when  the  tip  tank  is  abeam  the  carrier  ramp.  Throttle  back  to  establish 
a  descent  rate  of  approximately  400  fpm  but  maintain  15  units  AOA.  At  the  90° 
position  (see  Figure  A-l)  you  should  be  close  to  400  feet  of  altitude.  From 
that  point  continue  the  turn  to  roll  out  on  the  glideslope  and  on  the  extended 
centerline  of  the  landing  deck.  You  should  be  about  3000  feet  from  the 
carrier  (20  to  25  seconds  out  to  touchdown)  at  roll  out.  Continue  the 
approach  to  touchdown. 

For  straight-in  approach,  the  simulator  will  be  initialized  in  the  Landing 
configuration  two  miles  from  the  ramp,  15  units  Angle  to  Attack,  400  feet 
altitude  and  left  of  the  center  line.  This  is  where  you  will  start  your 
straight-in  approach. 

Upon  release  you  will  fly  the  aircraft  straight  and  level  and  maintain  400 
feet  altitude  (approximately  86  percent  power).  Flying  this  configuration  you 
will  make  a  gradual  cut  into  the  extended  center  line  of  the  landing  deck.  In 
addition,  by  maintaining  a  400  feet  altitude  you  will  intercept  the  glideslope 
and  a  centered  meatball  at  approximately  4500  feet  from  the  ramp.  When  the 
meatball  approaches  centerbal  1  you  are  to  reduce  power  to  83-84  percent  and 
continue  the  approach  to  touchdown. 

THE  CARRIER  APPROACH 

Precise  aircraft  control  is  essential  in  a  carrier  approach.  Vertical 
displacement  errors  at  the  ramp  (threshold  of  the  landing  deck)  of  a  few  feet 
can  be  disastrous,  as  can  descent  rate,  airspeed  or  attitude  errors  at 
touchdown.  Thus,  the  pilot  must  maintain  a  precise  glideslope  (generally  set 
at  3.5°)  and  maintain  the  correct  descent  rate,  airspeed  and  attitude. 
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CARRIER  LANDING  PATTERN 

MIRROR  APPROACH 


Figure  A-l.  Carrier  Landing  Pattern  from  the  Downwind  Leg 
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Conventional  landings  permit  some  deviations  in  these  parameters  but  Navy 
carrier  pilots  must  establish  them  early  in  the  approach  and  maintain  them  to 
touchdown.  Neither  is  it  acceptable  for  a  Navy  pilot  to  fly  a  loose  early 
approach  with  the  aim  of  establishing  better  control  near  the  carrier.  The 
potentially  disastrous  consequences  of  errors  makes  the  uncertainty  associated 
with  this  type  of  behavior  quite  unacceptable.  In  this  experiment  you  will 
learn  some  of  the  skills  needed  for  carrier  landings. 

PARAMETERS  FOR  APPROACH  CONTROL 

In  making  an  approach  from  the  roll  out  position  the  carrier  pilot  must  oe 
concerned  with: 

1)  current  position  in  relation  to  the  glideslope, 

2)  current  descent  rate — is  it  correct,  if  not  is  it  taking  him  away 
from  the  gl ideslope 

3)  airspeed  and  pitcn  attitude — integrated  into  one  instrument  known  as 
the  Approach  (Angle  of  Attack)  Indexer,  and 

4)  lineup. 

GLIDESLOPE  POSITION 

Glideslope  guidance  is  normally  given  by  the  Fresnel  Lens  Optical  Landing 
System  (FLOLS).  We  have  simulated  this  system  with  two  horizontal  bars  (to 
represent  the  datum  bars)  and  a  moving  dot  (referred  to  as  the  ball  or  the 
meatball).  The  system  is  illustrated  in  Figure  A-2  and  Figure  A-3  a  to  e.  A 
center  ball  indicates  that  the  aircraft  is  on  the  glideslope  (later  discussion 
will  note  that  correct  aircraft  attitude  is  necessary  for  that  to  be  true).  A 

high  ball  indicates  that  the  aircraft  is  above  glideslope,  and  a  low  ball  that 

it  is  below  glideslope.  At  two  balls  low  the  meatball  starts  to  flash.  Plus 
or  minus  two  balls  is  the  maximum  effective  range  of  the  system.  The  ball 
will  be  lost  off  the  top  or  the  bottom  at  larger  deviations  from  glideslope. 

A  real  FLOLS  projects  cones  of  light  from  the  ship  as  shown  in  Figure 
A-4,  Thus  the  system  is  angular.  Larger  errors  are  required  far  from  the 
ship  to  see  meatball  movement  than  are  required  near  the  ship.  At  3/4  mile  a 
12  foot  glideslope  displacement  is  needed  to  move  the  ball  off  center  while  at 
the  ramp,  a  one  foot  displacement  will  move  the  ball  off  center.  The  range  of 

the  FLOLS  is  approximate! y  +  3/4*  (precisely  +  47.5' )  or,  if  set  for  a  3.5° 

glideslope,  from2.75°  to  4.25°  (approximately) . 

CARRIER  LANOINGS 

In  making  a  carrier  landing  the  pilot  attempts  to  follow  the  FLOLS  center 
beam  to  the  deck  of  the  carrier.  If  he  can  maintain  a  center  ball,  and  keeps 
the  aircraft  in  the  correct  pitch  attitude,  a  hook  fixed  to  the  tail  of  the 
aircraft  (Figure  4)  will  follow  a  glide  path  that  is  parallel  to,  but  lower 
than  the  center  FLOLS  beam.  It  is  intended  that  the  hook  contact  the  deck 
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Two  balls  low 


Figure  A-3.  The  Fresnel  Lens  Optical  Landing  System 
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Figure  A-4.  Carrier  Approach  Schematic  Depicting  FLOLS  Envelope, 
Tail  Hook  Glide  Path,  and  Arrestment  Wire  Locations. 
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midway  betwen  the  second  and  third  of  four  cables  stretched  across  the  deck 
(these  cables  are  known  as  arrestment  wires).  The  hook  travels  forward  from 
this  point  to  snag  the  third  wire,  and  so  the  aircraft  is  halted. 

If  the  pilot  is  slightly  low  on  the  approach  he  may  snag  the  first  or 
second  wires.  If  he  is  very  low  (actually  an  error  of  10  feet  may  be  enough) 
he  may  hit  the  ramp,  thereby  bringing  disgrace  and  physical  harm  to  himself, 
and  severely  damaging  a  multi-million  dollar  aircraft.  If  a  pilot  is  slightly 
high  on  the  approach  he  may  snag  the  fourth  wire.  If  higher  (possibly  only 
two  feet  higher  than  optimum  he  may  miss  the  wires  altogether  and  fly  off  the 
end  of  the  carrier.  A  missed  approach  of  this  type  is  called  a  bolter. 
Fortunately  bolters  do  no  lasting  damage  (about  5  percent  of  approaches  result 
in  bolters),  but  they  do  detract  from  shipboard  efficiency.  Thus  the  ability 
to  follow  the  glideslope  contributes  to  a  Navy  pilot's  health,  happiness  and 
self-esteem. 

DESCEND  RATE 

The  aircraft  has  a  Vertical  Speed  Indicator  (VSI)  (Figure  A-5) ,  with  hash 
marks  shown  at  200  fpm  intervals  (Figure  A-6).  The  reference  descent  rate  for 
the  T-2  in  the  configuration  that  you  will  be  flying  is  515  fpm.  That  is,  if 
the  aircraft  is  on  the  glideslope  and  with  the  correct  attitude  and  airspeed, 
it  will  stay  on  the  glideslope  if  the  reference  descent  rate  is  maintained. 

If  you  are  above  glideslope  you  will  need  to  establish  a  descent  rate  of 
up  to  800  fpm,  while  if  you  are  below  glideslope  you  will  need  to  establish  a 
descent  rate  of  as  low  as  200  fpm.  These  corrections  will  return  you  to  the 
glideslope  at  an  appropriate  rate.  Maximum,  minimum,  and  optimum  vertical 
speeds  are  indicated  in  Figure  A-6. 

Note  that  if  you  perceive  an  incorrect  vertical  speed,  it  will  probably 
not  be  sufficient  merely  to  correct  back  to  the  reference  rate  (515  fpm)  even 
if  you  are  on  glideslope.  By  the  time  your  correction  has  taken  effect  you 
will  probably  be  off  glideslope  and  will  need  to  correct  in  a  direction 

opposite  to  that  which  caused  the  error.  The  techniques  for  correcting 

glideslope  errors  are  central  to  good  carrier  landings,  and  will  be  discussed 
in  detail  in  a  later  section. 

Descent  rate  information  can  also  be  obtained  from  the  meatball.  If  you 
have  a  center  ball,  but  see  it  moving,  you  can  judge  that  your  descent  rate- is 
incorrect.  In  addition,  if  you  are  high,  you  need  to  start  the  ball  moving 
down,  and  if  low,  start  it  moving  up.  You  can  use  the  rate  of  ball  movement 
to  establish  an  appropriate  corrective  descent  rate.  This  can  be  useful 
because  it  means  that  you  do  not  have  to  look  inside  the  cockpit  at  the  VSI. 

Unf ortunately,  it  is  possible  to  discern  movement  of  the  meatball  only  when 

the  aircraft  is  approximately  1500  feet  from  the  ship.  At  greater  distances 
the  rate  of  movement  is  so  low  that  it  is  below  the  threshold  for  the 
psychological  process  that  interprets  changes  in  position  as  rates  of 
movement.  Thus,  you  will  have  to  rely  on  the  VSI  until  you  close  on  the 
carrier. 
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To  assist  your  rate  judgments  we  have  added  some  vertical  arrows  to  tne 
FLOLS  as  shown  in  Figure  A-7.  They  are  calibrated  to  indicate  whether  you 
should  modify  your  vertical  speed.  A  null  indication  while  you  have  a  center 
ball  indicates  that  you  are  on  glideslope  and  staying  there.  Arrows  up  or 
down  indicate  that,  although  you  may  now  have  a  center  ball,  you  will  soon  be 
high  or  low.  If  you  are  above  or  below  glideslope,  a  null  indication  shows 
that  you  are  returning  to  the  glideslope  at  an  appropriate  rate.  Oown  arrows 
mean  you  are  descending  too  quickly.  Up  arrows  indicate  you  are  not 
descending  quickly  enough. 

If  you  are  high,  up  ^-rows  indicate  that  you  are  not  returning  to  the 
glideslope  quickly  enough,  and  could  even  be  going  further  from  it.  You 
should  descend  more  quickly.  Oown  arrows  indicate  that  you  are  returning  to 
the  glideslope  too  quickly  and  will  probably  over  shoot.  Reduce  your  descent 
rate. 

For  a  low  meatball  the  interpretations  are  just  the  opposite,  down  arrows 
indicate  that  you  are  not  returning  to  the  glideslope  quickly  enough  and  may 
even  be  flying  further  from  it,  while  up  arrows  indicate  you  are  approaching 
it  too  quickly  and  will  overshoot. 

The  basic  rule  is  to  null  the  arrows  wherever  you  are.  Up  arrows  indicate 
you  are  not  descending  quickly  enough.  Down  arrows  mean  you  are  descending 
too  quickly. 

The  arrows  will  be  available  during  your  initial  training,  but  they  will 
not  be  available  in  a  later  test  session.  Use  them  for  guidance,  but  do  not 
rely  on  them  at  the  expense  of  the  other  rate  information.  Use  the  arrows  to 
help  you  learn  to  use  the  other  rate  indications. 

ANGLE  OF  ATTACK 

The  FLOLS  is  a  passive  optical  system,  and  the  pilot  sees  a  center  ball 
when  his  eye  is  in  the  center  beam.  The  center  beam  is  set  so  that,  at  the 
correct  aircraft  atitude,  the  tail  hook  of  the  aircraft  is  proceeding  on  a 
glide  path  of  its  own,  towards  a  point  on  the  carrier  deck  midway  between  the 
second  and  third  arrestment  cable  (Figure  A-4) .  However,  the  hook  is  at  the 
other  end  of  the  aircraft  from  the  pilot's  eye,  and  simple  geometry  would 
suggest  that  an  incorrect  pitch  attitude  will  move  the  hook  above  or  below  its 
glide  path  even  when  the  pilot's  eye  is  on  the  correct  FLOLS  glideslope.  In 
fact,  the  hook  is  the  critical  point  of  the  aircraft  for  touchdown  accuracy, 
not  the  eye  of  the  pilot.  The  only  means  the  pilot  has  of  ensurinq  that  the 
hook  is  in  the  correct  position  is  by  following  the  FLOLS  beam  with  his  eye, 
and  flying  the  correct  AOA  (which  will  ensure  correct  pitch  attitude). 
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(a)  One-ball  high;  returning  to  the  reference  glideslope  at  an 
appropriate  rate. 


(b)  One-ball  high;  not  returning  to  the  glideslope  quickly 
enough  (may  even  be  going  higher). 


(c)  One-ball  high;  returning  to  the  glideslope  too  quickly 
and  will  probably  fly  through  it. 


Figure  A-7.  Three  Types  of  Indications  from  tne  Rate  Arrows. 
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To  monitor  AOA  the  Navy  pilot  is  provided  an  instrument  called  the 
Approach  Indexer.  You  will  find  it  above  and  to  the  left  of  the  instrument 
panel  (Figure  A-5).  It  consists  of  an  upper  and  lower  chevron  and  a  center 
circle  (donut).  It  is  possible  for  one  chevron,  or  the  donut,  or  a 
chevron-donut  pair  to  be  illuminated.  The  readings  and  their  interpretations 
are  shown  in  Figure  A-8. 


Correct  AOA  (15  Units) 
i.e.,  on  speed,  correct  attitude 

/  ' 

I  ' 

/  V 

14.5  to  15.5  units 


15.5  to  16.0  16+  units 


High  AOA  (more  than  15  Units) 
i.e.,  slow,  with  high  pitch  attitude 


14.0  to  1 4.5  1 4.0  units 


Low  AOA  (less  than  15  Units) 
i.e.,  fast,  with  low  pitch  attitude 


Figure  A-8.  Indications  from  the  Approach  (Angle  of  Attack)  Indexer 


A  chevron-donut  pair  can  generally  be  regarded  as  acceptable.  This  would 
allow  a  range  of  15+1  units.  More  extreme  AOA  errors  should  be  corrected  as 
is  described  in  a  later  section  of  this  reading. 
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LINEUP 

In  carrier  landings  the  pilot  lines  up  with  the  extended  center  line  of 
the  landing  deck.  Note  that  the  landing  deck  is  canted  at  10.5°  to  the 
longitudinal  axis  of  the  ship.  It  is  not,  therefore,  appropriate  to  use  the 
carrier  wake  or  the  main  deck  for  lineup.  Lineup  errors  are  corrected  with 
small  banking  turns  to  the  left  or  right.  You  will  need  to  use  fine  control 
pressures  in  moving  the  stick  to  the  left  or  right,  and  on  the  rudder  pedals, 
to  start  these  turns.  In  turning  onto  the  center  line,  you  should  anticipate 
closing  on  it:  that  is,  start  your  lineup  turn  before  you  reach  it.  If  you 
start  your  lineup  turn  when  you  reach  the  center  line,  you  will  find  yourself 
a  long  way  past  it  by  the  time  you  are  heading  the  simulator  in  the  right 
direction. 

At  night  you  will  need  to  use  the  drop  lights  at  the  stern  of  the  carrier 
to  assist  you  with  line  up.  If  you  are  lined  up  it  will  appear  as  a  straight 
extension  of  the  center  line  of  the  landing  deck.  If  you  are  off  center  it 
will  appear  angled  to  the  center  line.  It  will,  in  fact,  form  a  V  with  the 
center  line,  with  the  apex  of  the  V  pointing  in  the  direction  you  must  go  to 
line  up  (Figure  A-9) . 

ERROR  CORRECTION:  GLIDESLOPE  AND  AOA 


Upon  reaching  the  90  degree  position  (about  halfway  through  the  turn)  and 
acquiring  the  ball,  the  aircraft  is  on  the  glideslope.  Due  to  ever-i ncreasi ng 
closure  rate  on  the  touchdown  point,  the  rate  of  descent  required  to  maintain 
a  centered  ball  from  the  90  to  wings  level  on  final  is  an  ever-increasing 
amount.  Therefore,  less  power  may  be  required  from  the  90  to  the  start  of  the 
final  so  as  to  maintain  a  centered  ball,  while  the  nose  attitude  is  adjusted 
to  maintain  15  units  angle  of  attack.  In  addition,  you  will  need  to  reduce 
power  when  you  roll  your  wings  level  to  compensate  for  the  increased  lift. 

Always  keep  in  mind  your  glideslope  position  (i.e.,  meatball  position), 
your  vertical  speed  (noted  from  the  VSI  and  the  rate  of  movement  of  the 
meatball),  and  your  AOA.  Try  to  determine  a  reference  power  level  that  will 
maintain  you  on  the  glideslope.  The  location  of  the  left  and  right  power 
gauges  is  shown  in  Figure  A-5,  while  Figure  A-10  shows  approximate  optimum, 
maximum,  and  minimum  values.  Also,  note  on  Figure  10  that  each  minor  hash 
mark  (small  guage,  top  center)  represents  1  percent  of  power  and  major  hash 
marks  represent  10  percent  of  power.  A  reference  power  of  about  83  percent 
should  work  well.  Lead  corrections  with  power  (except  as  noted  in  2)c) 
below);  changes  of  2  percent  to  4  percent  should  be  sufficient.  Certainly  do 
not  go  above  90  percent  or  below  75  percent.  Follow  with  small  pitch  changes 
to  correct  or  maintain  AOA.  An  8.5°  pitch  up  is  correct;  and  corrections  for 
AOA  should  not  require  pitch  movements  to  below  7°  or  above  10°  (the  dot  on 
the  attitude  indicator  corresponds  to  1°).  Greater  changes  than  that  will 
indicate  that  you  are  overcontrolling  in  pitch. 
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Diagram  of  T-2C  power  gauqe, 
snowing  hash  marks  -  note  the 
needle  at  S3  percent,  and 
dotted  needles  at  90  percent 
and  75  percent . 


Figure  A-10.  Power  Gauge  (Sotn  cert  ana  Pi aht  Gauges  are  Identical). 


Remember  tnat  corrections  are  almost  always  started  with  a  power 
adjustment  and  AOA  errors  snould  generally  be  corrected  before  glideslope 
errors  (except  as  noted  in  2 ) c )  below).  "The  power  adjustments  for  a 
correction  will  be  made  in  three  vand  sometimes  four)  steps.  First  increase 
or  decrease  power  to  initiate  tne  correction.  Secondly,  take  out  tne 
correction  as  you  approach  the  correct  AOA  or  glideslope  position.  In  taking 
out  the  correction  go  past  your  reference  power  to  null  any  acceleration  or 
unwanted  velocity  component  that  you  have  introduced  in  the  first  step.  The 
third  step:  to  return  the  power  to  its  reference  level,  follows  the  second 
step  almost  immediately. 

If  you  need  to  make  a  large  power  correction  for  a  glideslope  error,  you 
may  find  it  necessary  to  insert  another  power  adjustment  between  the  first  and 
second  steps.  After  the  initial  correction  you  should  look  for  a  target 
descent  rate  that  will  return  you  to  the  glideslope  quickly  enouqh,  but  not  so 
quickly  that  you  will  not  be  able  to  stop  on  the  glideslope.  You  may  achieve 
the  target  descent  rate  before  you  near  the  glideslope.  If  so,  you  should 
take  out  some  of  your  power  correction  (probably  about  half)  so  that  you  do 
not  go  past  your  tarqet  descent  rate.  Specific  types  of  errors  are  discussed 
below. 

1)  AOA  errors 

If  on  glideslooe  and  correct  vertical  speed, 

a)  high  AOA  (slow):  add  power,  smoothly  push  the  stick  forward 
(slightly)  to  correct  AOA;  as  aircraft  accelerates  reduce  power 
to  slightly  less  than  reference  level,  and  then  almost 
immediately  adjust  back  to  reference  level. 

b)  low  AOA  (fast):  decrease  power,  smoothly  pull  stick  (slightly) 
to  correct  AOA;  as  aircraft  decelerates  increase  power  to 
slightly  higher  than  reference  level  and  then  almost  immediately 
decrease  power  to  reference  level. 
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2)  G1 ideslope  errors 

Note  that  if  your  AOA  is  correct  and  you  add  power  to  make  a 
glideslope  correction,  you  will  need  to  pull  the  stick  back  slightly 
to  maintain  the  correct  AOA  (because  with  the  same  stick  pressure  the 
extra  surge  of  power  will  push  the  aircraft  a  little  fast-.r  and  tend 
to  lower  its  attitude).  If  you  decrease  power  you  will  need  to  push 
the  stick  forward  slightly  to  maintain  AOA. 

a)  Going  high:  decrease  power  (if  AOA  is  low  the  decrease  in  power 
will  tend  to  correct  the  AOA  error  before  it  corrects  the 
glideslope  error;  otherwise  you  need  to  push  the  stick 
forward).  When  you  see  that  you  have  started  back  to  the 
glideslope  add  about  half  the  power  you  have  taken  out.  As  you 
near  the  glideslope  add  more  power  so  that  the  power  level  is 
now  slightly  above  the  reference  level.  Almost  immediately 
reduce  power  to  the  reference  level. 

b)  Going  low:  increase  power  to  start  the  ball  moving  up  (if  AOA 
is  high,  the  increase  in  power  will  tend  to  correct  the  AOA 
error,  but  let  the  ball  start  moving  up  before  you  ensure  tnat 
AOA  is  closing  on  the  correct  value).  When  you  see  that  you 
have  started  back  to  the  glideslope,  take  out  about  half  tne 
power  you  have  added.  As  you  near  the  glideslope  take  out  more 
power  so  that  the  power  level  is  now  slightly  below  the 
reference  level.  Almost  immediately  increase  power  to  the 
reference  level. 

c)  Correcting  for  a  low  or  a  high  in  close  (less  than  1000  ft  from 
touchdown):  for  a  low  add  power  to  start  the  ball  moving  up. 
Stop  the  ball  moving  up  by  adjusting  the  pitch  (this  is  the  only 
time  that  pitch  should  lead  power  in  making  an  adjustment).  Use 
power  to  get  back  on  speed. 

If  the  ball  is  moving  up  in  close  or  has  stopped  with  a  high 
indication  in  close  (either  as  a  result  of  an  overcorrection 
from  a  low,  a  slightly  low  descent  rate  from  farther  out,  or  for 
some  other  reason),  do  not  recenter. 

A  correction  at  this  point  can  lead  to  an  excessive  descent  rate 
at  touchdown  (correction  for  a  high  ball  in  close  can  produce  a 
5°  glideslope).  If  the  ball  develops  a  rapid  motion  towards  the 
bottom  of  the  lens,  apply  enough  power  to  stop  the  movement. 

LANDING  SIGNAL  OFFICER 

In  real  carrier  approaches  a  Landing  Signal  Officer  ( LSO )  is  stationed  to 
the  side  of  the  landing  deck  and  advises  the  approaching  pilot  by  radio  on  the 
suitability  of  his  approach.  He  may,  for  example,  advise  the  pilot  that  he  is 
high  or  low,  or  to  the  left  or  right.  He  may  give  instructions,  such  as 
"POWER"  to  indicate  that  the  pilot  should  add  power.  He  may  instruct  the 
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pilot  to  discontinue  his  approach,  and  go  around  to  set  up  for  another 
approach  by  flashing  two  vertical  light  arrays  on  the  FLOlS  and  callina 
"WAVE  OFF". 


The  role  of  the  LSO  is  also  instructional,  in  that  he  will  make  a  record 
of  the  pilot's  performance,  and  use  this  in  a  debrief  to  point  out  errors,  and 
to  advise  him  on  how  to  improve  his  approaches. 

In  this  experiment  we  have  a  computerized  LSO  to  give  selected  calls 
during  the  approach.  The  calls  are  listed  in  Table  A-l,  together  with  the 
type  of  error  that  will  evoke  the  call,  and  the  corrective  action  required. 

The  instructional  role  of  the  LSO  will  be  filled  by  an  experimenter  who 
has  been  trained  by  an  LSO  for  this  task.  He  will  comment  on  the  significant 
features  of  your  approach  at  the  end  of  each  trial,  and  will  suggest  ways  to 
improve.  These  suggestions  will  not  cover  new  material.  Anything  that  should 
be  explained  to  you  already  has  been  explained.  The  LSO  -  experimenter's 
comments  will  be  taken  from  this  briefing,  and  will  serve  to  remind  you  of  the 
material  covered,  and  to  orient  you  towards  the  errors  that  you  are  maxing  and 
the  appropriate  corrective  action.  Common  terminology  that  miqht  be  used 
during  these  instructions  is  shown  in  Table  A-2. 

SUMMARY 

It  requires  care  and  effort  to  learn  the  control  techniques  for  carrier 
landings.  Navy  pilots  complete  more  than  100  approaches  in  a  simulator  or  to 
a  shore-based  landing  strip  before  they  attempt  a  carrier  landing.  Our 
research  indicates  that  even  after  hundreds  of  carrier  landings  pilots 
continue  to  improve  their  glideslope  control.  We  will  be  measuring  your 
performance  throughout  the  trial,  not  just  at  the  deck  of  the  carrier.  Follow 
the  recommended  procedures,  and  in  particular  try  to  set  yourself  on  the 
glideslope,  and  with  the  correct  AOA  early  in  the  approach.  Your  errors  along 
the  glideslope  will  be  assessed.  Avoid  the  temptation  to  correct  by  leading 
with  pitch  adjustments.  Also  avoid  the  temptation  to  trap  a  wire  at  all 
costs.  If  you  are  high  as  you  approach  the  wires,  accept  it.  A  sudden  dive 
for  the  deck  at  this  point  will  downgrade  your  overall  ratinq  for  that 
approach  more  than  will  a  bolter.  You  should  approach  the  ,_sk  with  care  and 
pnrseverence .  Review  this  lesson,  and  note  the  feedback  during  the  trials. 
There  is  something  to  learn  from  even  a  bad  performance. 
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TABLE  A-l.  LSO  TRANSMISSIONS,  THEIR  MEANING, 
AND  REQUIRED  CORRECTIVE  ACTION 


TRANSMISSION 

MEANING 

REQUIRED  RESPONSE 

"YOU'RE  A  LITTLE 

A/C  is  between  .5 

Adjust  altitude  to  a  centered 

HIGH" 

and  1.5  meatbal Is 
above  gl ideslope. 

meatball  immediately. 

"YOU'RE  HIGH" 

A/ C  is  1.5  meatball s 
or  more  above 
g  1  ideslope. 

Reduce  power  and  adjust 
altitude  to  a  centered  meatball 
immediately. 

"YOU'RE  GOING  HIGH" 

A/C  is  less  than  .5 
meatbal 1 s  above 
glideslope  and  sink 
rate  is  less  than  60 
ft/mi n. 

Reduce  power  and  re-establish 
rate  of  descent. 

"YOU'RE  A  LITTLE 

A/C  i s  between  .5  and 

Maintain  current  altitude 

LOW" 

1  meatbal 1  below 
g  1  ideslope. 

until  glideslope  is  intercepted 

"YOU'RE  LOW" 

A/C  i s  more  than  1 
meatball  below 
g  1  ideslope. 

Add  power  and  adjust  altitude 
to  a  centered  meatball 
immediately. 

"YOU'RE  GOING  LOW" 

A/C  is  less  than  .5 
meatball  below 
glideslope  and  sink 
rate  i s  greater  than 

660  f t/mi n. 

Add  power  and  re-establish 
rate  of  descent. 

"A  LITTLE  POWER" 

A/C  is  between  .5  and 

1  meatbal  1  below 
gl ideslope. 

..dd  1  to  2  percent  power  to 
adjust  altitude  to  a  centered 
meatball  immediately. 

"POWER" 

A/C  i s  more  than  1 
meatball  below 
gl  ideslope 

Add  power  to  adjust  altitude 
to  a  centered  meatball. 

or 


A/C  is  in-close,  more 
than  .5  meatballs  below 
glideslope  and  sink 
rate  is  greater  than 
480  ft/mi n. 
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TABLE  A-l .  LSO  TRANSMISSIONS,  THEIR  MEANING, 

AND  REQUIREO  CORRECTIVE  ACTION  (cont'd) 


TRANSMISSION  MEANING  REQUIRED  RESPONSE 

"MORE  POWER"  Response  to  an  initial  Add  more  power. 

"power  command"  was 
inap propriate. 

"YOU'RE  FAST"  Angle  of  Attack  is  Correct  Airspeed/Angle  of 

less  than  13  units  Attack  Indication, 

and  sink  rate  is 
between  210  and  390 
ft/mi  n. 

"YOU'RE  SLOW"  Angle  of  Attack  is  Correct  Airspeed/Angle  of 

greater  than  16  units  Indication  with  power  addition, 
and  sink  rate  is 
between  480  and  660 
ft/mi n. 

“FLY  THE  BALL"  A/C  is  "in-close"  and  Fly  and  use  the  meatball  for 

more  than  1  ball  above  rate/altitude  information, 
gl ideslope. 

or 

A/C  is  "in-close"  and 
sink  rate  is  less  than 
210  ft/mi n. 
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TABLE  A-2.  COMMON  TERMINOLOGY 


1.  The  180°  position  -  A  position  on  the  downwin  leg  where  the  initial  turn 
onto  the  base  leg  is  commenced. 

2.  The  90°  position  -  A  position  reached  halfway  along  the  180°  arc  from  the 
"130"  to  the  landing  line. 

3.  Final  approach  -  That  portion  of  the  pattern  flown  from  the  sighting  of 
the  meatball  to  touchdown. 

4.  Groove  -  That  portion  of  the  final  approach  which  coincides  wit;,  the 
landing  line.  It  commences  upon  rolling  the  wings  level  with  tne 
aircraft  on  the  landing  line  and  allows  for  approximately  a  18-25  second 
strai  ghtway . 

5.  Cocked  up  -  Flying  too  slowly  or  at  too  high  an  angle  of  attack,  causing 
the  use  of  excessive  power  to  maintain  altitude  or  rate  of  descent.  This 
is  a  condition  that  exists  when  operating  on  the  back  side  of  the  power 
curve . 

6.  Oi ve  for  the  deck  -  Pushing  the  nose  over  and  establishing  an  excessive 
rate  of  descent.  This  causes  either  a  three-point  landing  (all  gear 
hitting  the  deck  at  the  same  time)  or  possible  nose  wheel  first. 

7.  Ramp  -  The  after  end  of  the  flight  deck  or  the  downwind  end  of  the 
platform  of  the  runway. 

8.  Bolter  -  A  touchdown  onboard  the  carrier  in  which  the  arresting  hook  does 
not  engage  an  arresting  wire,  usually  caused  by  landing  past  the  wire 
area  or  by  the  hook's  skipping  over  the  arresting  wires. 

9.  Meatbal 1  -  Terminology  used  to  describe  the  mirror  presentation  of  the 
source  lights  as  seen  by  the  pilot. 

10.  Clara  -  A  term  used  to  signify  that  the  meatball  has  not  been  sighted. 
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APPENDIX  B 

ANALYSIS  OF  VARIANCE  SUMMARIES 
FOR  TRAINING  TRIALS 


TABLE  B-l .  ANALYSIS  OF  VARIANCE  FOR  TOUCHDOWN 
WIRE  ACCURACY  TRAINING  SCORES 


LEVELS  Mean 


Source  of  Variance 

Hi  qh 

Low 

df 

Diff erence^ 

F 

Field  of  View 

Wide 

Nar 

1 

5.8 

(3.7)2 

4. C  ' 

Scene  Detai 1 

Day 

Night 

1 

3.7 

(1.5) 

1.65 

Motion 

On 

Off 

1 

-2.9 

(-) 

1.03 

Approach  Type 

St.  In 

Circ 

1 

5.9 

(3.8) 

4.20 

FLOLS  Rate  Cue 

Cuing 

No  Cue 

1 

1.2 

(-) 

0.16 

Turbulence 

Calm 

Winds 

1 

6.4 

(4.5) 

4.96* 

Pilot  Type 

Nav  P-3C 

AF  T38 

1 

-6.9 

(5.1) 

5.63* 

FOV  x  App.  Type 

1 

(-) 

0.04 

S.DTL  x  App.  Type 

1 

(-) 

0.11 

2-Factor  Int  (No  Pi  1)3 

7)Re- 

(5.5) 

0.70 

2-Factor  Int  (Pi l)4 

6 )  S  i  d- 

(4.1) 

0.61 

2*3  Way  Strings 

9)ual 

(10.2) 

Blocks  (10  Trials) 

3 

(30.4) 

32.32** 

2-Factor  Int  (Blocks) 

21 

l7-5 

1.13 

3-Factor  Int  (Blocks) 

72 

(22.6) 

Grand  Mean 

27.0 

Std.  Err.  Difference 

2.9 

Std.  Deviation 

8.1 

iMean  of  observations  taken  under  high  level  minus  mean  of  observations  taken 
under  low  level  of  factor. 

Zval  ues  in  parentheses  are  percent  variance  accounted  for  in  the  experiment. 
Percents  less  than  1.0  are  shown  by  a  dash  (-). 

^Two-f actor  interactions  not  involving  pilot  type. 

^Two-factor  interactions  involving  pilot  type. 

*p  <  .05 

**p  <  .01 
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TABLE  8-2.  ANALYSIS  GF  VARIANCE  FOR  GLIDESLOPE 
TRACKING  TRAINING  SCORES 


Source  of  Variance 

LEVELS 

> 

Low 

df 

Mean 

Difference^- 

F 

Field  of  View 

Wide 

Nar 

1 

4.8 

(3.1)2 

1.97 

Scene  Detail 

Day 

Ni  ght 

1 

3.6 

(1.6) 

1.03 

Motion 

On 

Off 

1 

0.0 

(-) 

0.00 

Approach  Type 

St.  In 

Circ 

1 

9.4 

(11-9) 

7.48* 

FLOLS  Rate  Cue 

Cuing 

No  Cue 

1 

-3.7 

(1.8) 

1.11 

Turbulence 

Calm 

Winds 

1 

3.2 

(1.3) 

0.83 

Pilot  Type 

Nav  P-3C 

AF  T38 

1 

-5.2 

(3.7) 

2.30 

FOV  x  App.  Type 

1 

(-) 

0.25 

S.DTL  x  App.  Type 

1 

(-) 

0.41 

2-Factor  Int  (Mo  Pi  l)3 

7)Re- 

(6.5) 

0.42 

2-Factor  Int  (Pi  l)4 

6 ) s i d- 

(8.6) 

0.65 

2+3  Way  Strings 

9)ual 

(19.8) 

Blocks  (10  Trials) 

3 

(17.7) 

23.52** 

2-Factor  Int  (Blocks) 

21 

(5.0) 

0.96 

3-Factor  Int  (Blocks) 

72 

(18.0) 

Grand  Mean 

27.1 

Std.  Err.  Difference  3.3 

Std.  Deviation  9.4 


iMean  of  observations  taken  under  level  minus  mean  of  observations  taken 
under  low  level  of  factor. 

3Va  lues  in  parentheses  are  percent  variance  accounted  for  in  the  experiment. 
Percents  less  than  1.0  are  shown  by  a  dash  (-). 

3Two-factor  interactions  not  involving  pilot  type. 

4Two-f actor  interactions  involving  pilot  type. 

*p  <  .05 

**p  <  .01 
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TABLE  B-3.  ANALYSIS  OF  VARIANCE  FOR  LINEUP 
TRACKING  TRAINING  SCORES 


LEVELS  Mean 


Source  of  Variance 

High 

Low 

df. 

Difference^ 

F 

Field  of  View 

Wide 

Nar 

1 

-2.7 

(-)2 * 

0.14 

Scene  Detai 1 

Oay 

Ni  ght 

1 

4.4 

(-) 

0.35 

Motion 

On 

Off 

1 

-0.8 

(-) 

0.01 

Approach  Type 

St.  In 

Circ 

1 

15.3 

(10.4) 

4.23 

FLOLS  Rate  Cue 

Cuing 

No  Cue 

1 

0.0 

(-) 

0.00 

Turbulence 

Calm 

Winds 

1 

-1.8 

(-) 

0.06 

Pilot  Type 

Nav  P-3C 

AF  T38 

1 

10.9 

(5.3) 

2.15 

FOV  x  App.  Type 

1 

(-) 

0.14 

S.DTL  x  App.  Type 

1 

(-) 

0.07 

2-Factor  Int  (No  Pi  1)3 

7)Re- 

(16.1) 

0.79 

2-Factor  Int  (Pi  1) 4 

6 )  s  i  d- 

(11.7) 

0.67 

2+3  Way  Strings 

9)ual 

(26.3) 

Blocks  (10  Trials) 

3 

(5.2) 

8.04** 

2-Factor  Int  (Blocks) 

21 

(7.7 

1.71 

3-Factor  Int  (Blocks) 

72 

(15.4) 

Grand  Mean 

51.0 

Std.  Err.  Difference 

7.3 

Std.  Deviation 

20.6 

iMean  of  observations  taken  under 

high  level 

minus 

mean  of 

observations 

taken 

under  low  level  of  factor. 


2Va  lues  in  parentheses  are  percent  variance  accounted  for  in  the  experiment. 
Percents  less  than  1.0  are  shown  by  a  dash  (-). 

3Two-f actor  interactions  not  involving  pilot  type. 

4Two-f actor  interactions  involving  pilot  type. 

*p  <  .05 

**p  <  .01 
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TABLE  8-4.  ANALYSIS  OF  VARIANCE  FOR  ANGLE  OF  ATTACK 


TRACKING  TRAINING 

SCORES 

LEVELS 

Mean 

Source  of  Variance 

Hi  gh  Low 

df 

Difference! 

F 

Field  of  View 

Wide  Nar 

1 

6.0 

(4.2)2 

3.91 

Scene  Detail 

Day  Night 

1 

4.5 

(2.4) 

2.21 

Motion 

On  Off 

1 

3.0 

(1.0) 

0.97 

Approach  Type 

St.  In  Circ 

1 

8.0 

(7.2) 

6.80* 

FLOLS  Rate  Cue 

Cuing  No  Cue 

1 

-0.9 

(-) 

0.11 

Turbulence 

Calm  Winds 

1 

5.2 

(3.0) 

2.85 

Pilot  Type 

Nav  P-3C  AF  T38 

1 

9.8 

(10.9) 

10.24** 

FOV  x  App.  Type 

1 

(-) 

0.02 

S.DTL  x  App.  Type 

1 

(1.2) 

1.17 

2-Factor  Int  (No  Pi  1)3 

7)Re- 

(4.4) 

0.52 

2-Factor  Int  (Pi  1)4 

6 }  s  i  d- 

(8.1) 

1.13 

2+3  Way  Strings 

9)ual 

(10.9) 

Blocks  (10  Trials) 

3 

(17.0) 

18.49** 

2-Factor  Int  (Blocks) 

21 

(7.6 

1.18 

3-Factor  Int  (Blocks) 

72 

(22.0) 

Grand  Mean 

41.8 

Std.  Err.  Difference 

3.2 

Std.  Deviation 

8.9 

iMean  of  observations  taken  under  high  level 

1  minus 

mean  of 

observations 

taken 

under  low  level  of  factor. 


2va  lues  in  parentheses  are  percent  variance  accounted  for  in  the  experiment. 
Percents  less  than  1.0  are  shown  by  a  dash  (-). 

3lwo-f actor  interactions  not  involving  pilot  type. 

^Two-factor  interactions  involving  pilot  type. 

*p  <  .05 

**p  <  .01 
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